Mohsin et al. 2024 – Human-AI Collaboration in SOCs
Citation
Mohsin, A., Janicke, H., Ibrahim, A., Sarker, I. H., & Camtepe, S. (2024). A Unified Framework for Human–AI Collaboration in Security Operations Centers with Trusted Autonomy. arXiv:2505.23397.
Core Contribution
A structured framework for human-AI collaboration in Security Operations Centers that integrates AI autonomy, trust calibration, and human-in-the-loop decision-making through a five-level autonomy taxonomy mapped to SOC analyst tiers.
Framing
The authors argue that existing SOC frameworks treat automation as binary rather than graduated, lacking systematic structures to manage human oversight, trust calibration, and scalable autonomy. Their key insight: effective human-AI collaboration requires dynamic calibration across three interconnected dimensions, autonomy, trust, and human involvement, that adapt to task complexity and risk.
Key Concepts
The Triadic Model
Three core pillars of human-AI collaboration:
- Autonomy (A) – AI’s ability to act independently
- Human-in-the-Loop (H) – Degree of human oversight
- Trust (T) – Confidence in AI’s reliability and transparency
Formal Definitions
Autonomy: A = 1 − (λ₁C + λ₂R)(1 − T)
- C = task complexity
- R = risk
- T = trust
- Higher complexity/risk reduces autonomy; higher trust increases it
HITL: H = 1 − A
- Inverse relationship: as autonomy increases, human involvement decreases
Trust: T = α₁E + α₂P + α₃(1 − U)
- E = explainability
- P = performance history
- U = uncertainty
Five Autonomy Levels (adapted from SAE J3016)
| Level | Description | A Range |
|---|---|---|
| 0 | Manual operations | ≈0 |
| 1 | AI-assisted (decision support) | 0.2–0.4 |
| 2 | Semi-autonomous (AI acts with approval) | 0.4–0.6 |
| 3 | Conditionally autonomous (HITL) | 0.7–0.8 |
| 4 | Fully autonomous (minimal oversight) | 0.9–1.0 |
Three HITL Configurations
- Human-in-Control – Full HITL; humans make all decisions
- Human-on-the-Loop (HOtL) – AI executes under conditional oversight
- Human-out-of-the-Loop (HOoTL) – AI autonomous; humans audit/govern
Transferable Insights
- Graduated autonomy beats binary automation – Most frameworks assume on/off; real collaboration requires a spectrum
- Trust mediates autonomy – You can’t scale autonomy without building trust through demonstrated performance and transparency
- Inverse relationship formalization – H = 1 − A elegantly captures the tradeoff between human control and AI independence
- Complexity and risk constrain autonomy – Even with high trust, high-complexity/high-risk tasks require human involvement
- Trust calibration is dynamic – Trust should evolve through interaction, not be assumed
Case Study Findings
The ACDC CyberAlly deployment (LLM-based SOC assistant) showed:
- 50% reduction in false positives
- 67% faster investigations
- 80% reduction in mean time to respond
- Graduated trust: analysts initially verified everything, gradually accepted AI autonomy