When a language model answers a multi-hop question from retrieved documents, it usually writes out its intermediate steps first: "the director of film X is Y; Y was born in Z; so the answer is Z." This step-by-step text is called chain-of-thought reasoning. Readers, and the systems built on top of these models, assume the trace shows how the answer was reached. The retrieved documents feed the trace, and the trace feeds the answer. Two bodies of evidence cast doubt on that assumption. The first comes from work on retrieval-augmented generation, meaning models that answer by reading documents fetched from an external index. In that setting, up to 57% of a model's citations were attached after the fact by surface word matching. The second comes from closed-book reasoning: hidden states often carry the final answer before any reasoning text is written. Neither finding has been tested where the two meet. Nobody has checked whether, in a multi-hop retrieval chain, the evidence actually flows through the written hops. The answer could instead be computed directly from the documents, or from the model's memorized knowledge, with the trace written alongside as decoration.
We propose a two-sided intervention audit to answer that question per hop, per instance, and across model families. One side edits the evidence: we change the fact in the retrieved paragraph that supports hop $k$ and check whether the trace's stated intermediate answer and the final answer both follow the edit. The other side edits the trace: we keep the documents fixed, overwrite the model's own written answer at hop $k$, let it continue, and see whether the final answer follows the trace or reverts to the documents. Together the two edits place each hop in one of four cases. The evidence can pass through the trace. The trace can be bypassed while the answer still reads the documents. The trace can be load-bearing but driven by memory rather than evidence. Or the hop can be purely decorative. We run this on instruction-tuned, reasoning-trained, reasoning-distilled, and retrieval-specialized models, because prior work shows these families use their traces in very different ways.
Core Claim
Our claim is that faithfulness in retrieval reasoning has two separable parts, and existing tests measure at most one. The first part is evidence-following: does the trace report what the documents say? The second is trace-mediation: does the answer depend on what the trace says? A model can score well on one and fail the other. If so, citation-based and answer-level grounding checks will misjudge how trustworthy multi-hop retrieval reasoning is, in ways that differ by model family. The edits also create conflicts between documents and memory naturally, rather than through planted cues. That lets us test whether retrieved evidence can override an answer the model has already settled on, or whether the model cites the new evidence while still answering from memory.
This differs from the closest work in specific ways.
Correctness is not Faithfulness in Retrieval Augmented Generation Attributions inserts short answer phrases into documents and measures false citation shifts. It covers single-step citation only, on Command-R+ alone. It reports false citation rates of 57% for relevant uncited documents, 55% for documents cited elsewhere, and 12% for random documents. It tests neither multi-step traces nor whether the answer depends on the trace.
Chain-of-Retrieval Augmented Generation (CoRAG) trains Llama-3.1-8B to interleave sub-queries, sub-answers, and retrieval. It keeps training chains only when the final answer is correct, so it never checks whether its sub-answers carry the computation.
Analysing Chain of Thought Dynamics shows that the model's training type changes how much it relies on its trace. Distilled-reasoning models changed answers in 65% of cases, against 25% for instruction-tuned and 24% for reasoning models. But that study used fabricated metadata and authority cues on closed-book multiple-choice questions. We replace those cues with edits to retrieved evidence on open-ended multi-hop questions.
Research Program
The main testbed is MuSiQue, which builds 2–4 hop questions from single-hop ones. MuSiQue applies filters so the intermediate hops cannot be skipped. It also annotates which supporting paragraph belongs to which hop, so we can edit exactly one hop's evidence. Its human scores are 78.0 F1 on answers and 93.9 F1 on support identification. Each question comes with a fixed set of gold and distractor paragraphs, which lets us separate reasoning failures from retrieval failures. 2WikiMultihopQA serves as a replication set.
Evidence edits. We use two kinds.
- A leaf edit replaces the answer entity in the final hop's paragraph with a plausible entity of the same type. The correct counterfactual answer is then known exactly.
- A bridge edit rewrites hop $k$'s paragraph so the bridge entity becomes a different entity $B$. We then substitute a real Wikipedia paragraph about $B$ that states the next relation. We keep only instances where such a paragraph exists, so the counterfactual chain stays well defined.
Every edit is checked against the model's own closed-book answers to each single-hop sub-question. This lets us separate edits that contradict the model's memory from edits on facts the model does not know.
Trace edits. These follow the step-intervention idea of FRIT, which swaps style-matched distractor facts into reasoning steps. In the model's own natural trace, we replace the stated sub-answer at hop $k$ with a same-type entity. We truncate everything after it, keep the documents unchanged, and let the model resume. We use free-form traces and not "Step 1/2/3" templates. This matters because the Corruption-Probe audit found that templated scaffolds turned error detection into substring matching, inflating it from about 4% to over 95%.
Scoring. For hop $k$ of instance $i$, let $E_{ik} \in \{0,1\}$ indicate that, after an evidence edit, the trace's stated sub-answer and the final answer both move to the edited value. Let $T_{ik} \in \{0,1\}$ indicate that, after a trace edit, the final answer follows the edited trace rather than the unchanged documents. The four cells of $(E, T)$ give the classification above. The headline quantity for each model family is the mediated-grounding rate,
$$ M = \frac{1}{|H|}\sum_{(i,k)\in H} E_{ik}\,T_{ik}, $$
where $H$ is the set of edited hops. A high $M$ means the documents reach the answer through the trace. A high $E$ with low $T$ means the documents reach the answer by a path that skips the trace. That is the parallel prompt-to-answer route that How Likely Do LLMs with CoT Mimic Human Reasoning? describes for closed-book prompts.
White-box probe. For the open-weight models we add one internal measurement. Chain-of-Thought Reasoning In The Wild Is Not Always Faithful used linear probes to show that closed-book answers are often encoded before any reasoning token. We train the same kind of probe on hidden states at the last prompt token. The goal is to predict the eventual final answer and to see whether, under an evidence edit, the probe already points to the edited answer or to the memorized one. This directly tests whether retrieved evidence breaks a pre-computed answer. We do not use sparse-autoencoder features: RAGLens found they do not transfer across models, and they would undercut a cross-family comparison.
Model coverage. The cross-family scope is the point of the study, not an add-on. We evaluate:
- instruction-tuned models: Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, Mistral-7B-Instruct;
- a model with a built-in reasoning mode: Qwen3-8B in thinking mode;
- a distilled-reasoning model from the DeepSeek R1 family;
- a retrieval-specialized model trained CoRAG-style, using released weights if available or otherwise reproduced from the paper's recipe;
- Command-R+, to connect to the earlier single-model citation result.
Where compute allows we add a larger Qwen3 model. Counterfactual Simulation Training found that Qwen3-235B was not more faithful out of the box than Qwen3-4B, and we want to know whether that holds in the retrieval setting.
Use of the labels (conditional). If the audit finds that a large fraction of correct chains are bypassed, we test whether the labels are useful. We compare CoRAG-style rejection sampling that keeps chains by final-answer correctness alone against sampling that also requires $E\,T=1$ on a sampled hop, at the same number of kept training chains on Llama-3.1-8B. This phase is secondary. It runs only if the measurement shows a gap worth closing.
Baselines
Naive reference. We assume a trace is faithful whenever the final answer is correct and the trace mentions the gold supporting entities. This is how multi-hop retrieval systems are judged today, through exact match and support F1. The audit beats this reference if a substantial share of correct, entity-matching chains land in the bypassed or decorative cells. The share must be large enough that answer correctness and $M$ come apart in a rank comparison across models.
Phrase-injection citation test. This is the test from Correctness is not Faithfulness in Retrieval Augmented Generation Attributions, rerun on every model in our set, on the same MuSiQue questions, with citation prompting added. It is the strongest existing black-box faithfulness test for retrieval. Its reported reference values are false citation rates of 57% for relevant uncited documents, 55% for documents cited elsewhere, and 12% for random documents, all on Command-R+. Our test adds information if its per-instance verdicts disagree with the phrase-injection verdicts on a meaningful fraction of hops. We confirm the disagreements are real by checking that our trace-edit outcome predicts held-out behavior: whether an answer survives deletion of the cited paragraph. We also check that phrase injection fails to predict it.
LLM judge. The strongest existing detector is the "Faithful-Judge" from FaithCoT-Bench. FaithCoT-Bench reports F1 between 65 and 77 for it on closed-book traces, more than 30% above counterfactual and logit-based detectors. We give it the full trace and the documents, with the same prompt and judge model across all generator models. We then score it against our causal labels as ground truth. Logit-confidence detectors are included as a weak floor, since the same benchmark found them often below 50 F1. No result exists for this judge on retrieval-grounded multi-hop traces, so we have no prior figure to beat. The question is whether it reaches its closed-book range against causal labels.
Answer-level edit-following (ablation). This isolates the key ingredient. It runs only the evidence edits and asks whether the final answer follows the edited document, skipping the trace edits. This is the standard knowledge-conflict style of measurement. If its model ranking and per-instance verdicts match our full $M$, the trace-side intervention adds nothing, and we will say so.
Matched compute. All baselines see the same questions, paragraphs, decoding settings, and generator outputs. The judge receives the same number of calls per instance as our audit spends on edited reruns.
Why not just…?
Why not just rerun the phrase-injection test on more models? That alone would close the single-model gap in the citation paper, and we do it as a baseline. It still only measures whether citation labels respond to surface overlap. It cannot tell whether a trace that names the right document actually passes its content to the answer. Our trace edits measure exactly that. The comparison against phrase injection shows how often the two tests disagree.
Why not just ask an LLM judge whether the trace is faithful? This is the most serious practical threat, because the judge is cheap and was the best detector in FaithCoT-Bench. However, FaithCoT-Bench also reports that stronger models produce more fluent unfaithful traces that fool detectors. And the Corruption-Probe audit found models caught errors in their own corrupted traces only 4.1% of the time even with a hint. A judge reads text, while bypass is a fact about computation. If the judge's agreement with our causal labels turns out to be high across all families, the practical recommendation becomes "use the judge," and the audit becomes the tool that certified it.
Why not just measure whether answers follow edited documents? If the answer tracks the evidence, one might argue the trace is irrelevant. That argument fails for monitoring. A trace that is bypassed cannot be audited or corrected, even when the answer happens to be grounded. The answer-level ablation tests whether this distinction matters in practice. If $T$ is near 1 whenever $E$ is 1 for every family, the simpler measurement suffices and we report that.
Why not just use a reasoning-trained model and assume the problem is gone? Thinking models cut closed-book post-hoc rationalization sharply. DeepSeek R1 showed 0.37% and Claude 3.7 Sonnet (Thinking) 0.04% on anti-symmetric question pairs. But that result comes from settings without documents. Long traces may also make bypass easier to hide. The cross-family comparison answers this directly.
Proposed Experiments
The main experiment computes $E$, $T$, and $M$ per hop position and hop count on about 1,500 MuSiQue-Ans development questions for each model, using leaf and bridge edits. The four-cell breakdown is reported separately for edits that conflict with the model's memory and edits that do not. Results are replicated on 2WikiMultihopQA.
Task accuracy is reported alongside so that faithfulness is never read without competence. The only reported reference figures in the file are CoRAG's exceeding IterDRAG and Search-o1-32B by over 10 exact-match points, and its retrieval recall rising from 29.0% to 47.1%. No reported result was found on Lacuna for the absolute exact-match scores of these specific models on MuSiQue under fixed gold-plus-distractor context.
First decisive check: does mediation separate from grounding? We look for models whose ranking on $E$ differs from their ranking on $M$, or for a large bypassed cell in any family. If neither appears, the two-part view of faithfulness is unnecessary for this regime.
Second decisive check: is the family pattern different with documents than without? We compare each family's trace-reliance in our setting with the closed-book pattern from Analysing Chain of Thought Dynamics. If distilled models remain the most trace-dependent and instruction-tuned models the least, the closed-book findings carry over. If the ordering changes, retrieval alters how traces are used, and single-model retrieval studies are especially risky.
Third decisive check: can evidence break a pre-computed answer? On memory-conflicting edits, we measure how often the last-prompt-token probe points to the memorized answer while the trace cites the edited document. That is the post-hoc pattern specific to retrieval. Results are broken down by family.
Fourth decisive check: how well do cheap detectors track causal labels? We report the judge's and the logit baselines' F1 against our labels. If the faithfulness-filtered training phase runs, it counts as a success only if it raises $M$ on held-out questions without lowering exact match, compared at an equal number of kept chains.
References
- Correctness is not Faithfulness in Retrieval Augmented Generation Attributions
- Chain-of-Retrieval Augmented Generation
- Analysing Chain of Thought Dynamics: Active Guidance or Unfaithful Post-hoc Rationalisation?
- ♫ MuSiQue: Multihop Questions via Single-hop Question Composition
- FRIT: Using Causal Importance to Improve Chain-of-Thought Faithfulness
- Auditing Chain-of-Thought Faithfulness for Trustworthy AI: A Reproducible Corruption-Probe Protocol Across Eleven Frontier LLMs
- How Likely Do LLMs with CoT Mimic Human Reasoning?
- Chain-of-Thought Reasoning In The Wild Is Not Always Faithful
- Toward Faithful Retrieval-Augmented Generation with Sparse Autoencoders
- Counterfactual Simulation Training for Chain-of-Thought Faithfulness
- FaithCoT-Bench: Benchmarking Instance-Level Faithfulness of Chain-of-Thought Reasoning