Behzad et al. 2025
Full Title: Beyond Predictions: A Study of AI Strength and Weakness Transparency Communication on Human-AI Collaboration
Summary
User study (n=272) examining how different levels of AI performance information affect human-AI collaboration in an income prediction task. Tested five conditions: no AI helper, prediction only, prediction + 80% accuracy statement, prediction + confidence scores, and prediction + contextual awareness (how well the AI performs on similar cases).
Key Contributions
- Developed a “self-assessing AI” by training a decision tree on the primary model’s mistakes, enabling the system to identify and explain where it’s likely to err
- Found that any performance information improves task performance, but only confidence scores and contextual awareness significantly improved trust calibration
- Awareness-based feedback slightly outperformed confidence scores in reducing overreliance on incorrect-but-overconfident predictions
Framing Insight
The paper positions itself against two streams of research that have failed to deliver: explainability (making black-boxes interpretable) and accuracy disclosure (telling users about performance). Neither alone calibrates trust effectively. The insight is that granularity and context of performance information matter more than whether you provide it.
Extracted Content
Atoms:
- 05-atom—confidence-is-not-awareness
- 01-atom—calibrated-trust-vs-high-trust
- 01-atom—algorithm-aversion-definition
- 05-atom—overconfident-wrong-critical-case
Molecules:
Methodological Notes
- Income prediction task using ACS PUMS dataset (California 2018)
- Random Forest classifier (80% accuracy) as primary model
- Decision tree trained on balanced correct/incorrect labels to create the “flaw tree”
- Consistent information across conditions (high confidence ↔ high similar-case accuracy)