What Does Actionable Uncertainty Communication Look Like?
If transparency about AI uncertainty creates as many problems as it solves, what alternative approaches might enable appropriate reliance?
The research shows that labeling miscalibration doesn’t help users respond correctly. What would?
Potential directions:
- Confidence transformation: Show recalibrated confidence rather than labeling the miscalibration
- Comparative framing: “On tasks like this, you’ve historically been more/less accurate than the AI”
- Behavioral nudges: Interface elements that slow down acceptance of high-but-unreliable confidence
- Outcome feedback: Real-time tracking of when AI was right/wrong to build intuition
- Decomposed uncertainty: Separate “model confidence” from “calibration reliability” as distinct signals
Why this question matters:
Current practice defaults to “show a confidence percentage.” This paper suggests that’s harmful when the percentage is misleading, but the alternative (labeling the misleading nature) doesn’t work either. We need a third path.
Related: 01-molecule—calibration-transparency-principle, 07-molecule—ui-as-ultimate-guardrail