Confidence Calibration

A well-calibrated AI system’s confidence scores accurately reflect the true likelihood of being correct.

When an AI says it’s 70% confident across many predictions, it should be right about 70% of the time. The confidence score represents an honest estimate of correctness probability.

Overconfident systems state confidence higher than actual accuracy (e.g., says 80%, is correct 70%).

Underconfident systems state confidence lower than actual accuracy (e.g., says 60%, is correct 70%).

Well-calibrated systems have confidence that matches accuracy.

Calibration is technically challenging, many ML algorithms, especially deep learning models, produce systematically miscalibrated confidence. The increase in model capacity and lack of regularization correlate with worse calibration.

This is distinct from accuracy itself. A 70%-accurate model can be well-calibrated (if it reports 70% confidence), overconfident (reports 90%), or underconfident (reports 50%). Calibration is about honesty in uncertainty expression, not raw performance.

Related: 05-atom—uniform-confidence-problem, 05-atom—calibration-detection-gap