Faithfulness vs Correctness in RAG

Two evaluation targets that sound similar but measure different things:

Faithfulness measures whether the generated response accurately reflects what’s in the retrieved documents. The response should not contradict or hallucinate beyond its sources.

Correctness measures whether the generated response matches ground truth. The response should be factually accurate independent of what was retrieved.

A response can be faithful but incorrect, accurately summarizing documents that happen to be wrong. A response can be correct but unfaithful, providing accurate information the model pulled from parametric memory rather than the retrieved context.

This distinction matters operationally: faithfulness can be evaluated without knowing ground truth (comparing output to retrieved docs), while correctness requires external verification.

For high-stakes applications, both matter. A system that’s faithful to bad sources fails differently than one that hallucinates beyond good sources, and the failure modes require different interventions.

Related: 05-atom—uniform-confidence-problem, 04-atom—provenance-design