Human Agency and Oversight

The principle that AI systems should enhance human decision-making rather than replace it, with mechanisms for meaningful human intervention.

Three levels of implementation:

• Human-in-the-loop: Humans approve each decision before action
• Human-on-the-loop: Humans monitor and can intervene during operation
• Human-in-command: Humans set goals and constraints but don’t oversee individual actions

The appropriate level depends on stakes and reversibility. Healthcare diagnostics might require human-in-the-loop; content recommendations might accept human-in-command.

The challenge: as systems scale, maintaining meaningful oversight becomes harder. A human “approving” hundreds of decisions per hour isn’t exercising judgment, they’re providing a rubber stamp.

Related: 07-atom—automation-control-false-tradeoff, 07-molecule—ui-as-ultimate-guardrail