Hallucination from Ungrounded Reasoning

Chain-of-thought reasoning without external grounding produces hallucinations at scale. When LLMs reason using only their internal representations, they generate plausible-sounding but factually incorrect statements that then propagate through the rest of the reasoning chain.

In error analysis of HotpotQA tasks, 56% of chain-of-thought failures were attributed to hallucination, the model fabricated facts or reasoning steps. In contrast, when reasoning was interleaved with external information retrieval (Wikipedia lookups), hallucination dropped to 0% of failures.

The mechanism: pure reasoning is a “static black box” where the model cannot verify its own claims. Once a hallucinated fact enters the reasoning chain, subsequent steps build on it, compounding the error.

Related: 05-atom—reasoning-grounding-tradeoff, 05-molecule—thought-action-observation-pattern, 05-atom—uniform-confidence-problem