The Flexibility-Factuality Tradeoff

Pure reasoning and grounded reasoning represent a tradeoff, not a hierarchy.

Chain-of-thought reasoning is more flexible in formulating reasoning steps, it can make creative leaps, synthesize across domains, and explore hypotheticals. But this flexibility comes at the cost of factuality. Without external verification, the model hallucinates.

Grounded reasoning (interleaving thoughts with external lookups) is more factual and trustworthy, claims can be traced to sources. But the structural constraint of alternating thought-action-observation steps reduces flexibility. Error analysis shows 47% of grounded reasoning failures are “reasoning errors,” often the model getting stuck in loops or failing to formulate the right next step.

The implication: neither approach dominates. The best results come from switching between them based on confidence. When internal knowledge seems sufficient, reason freely. When uncertain, ground in external retrieval.

Related: 05-atom—hallucination-from-ungrounded-reasoning, 05-atom—sparse-vs-dense-reasoning