Evidence-Swap Auditing: Does a RAG Answer Change When Its Evidence Does?

Retrieval-augmented generation (RAG) means an LLM answers after reading documents fetched from a search index. A RAG answer can be correct, cite a document that really says the right thing, and still not have been produced from that document. The model may have answered from memory and attached the citation afterward. Such an answer looks fine to every current hallucination checker. It becomes a hallucination only when the documents and the model's memory disagree: after a corpus update, in a private knowledge base, or when the memory is simply wrong. We propose a direct test for this hidden failure. For each claim in a RAG answer, rewrite the specific fact in the retrieved documents that supports it, changing it to a plausible alternative everywhere it appears, then rerun the model. If the claim, and the reasoning step that led to it, move to the new value, the claim is causally grounded. If the claim stays put, it was never really coming from the evidence.

The test needs only text in and text out, so it works on closed APIs. It splits each answer into two links that can each break: whether the model's written reasoning follows the documents, and whether the final answer follows the reasoning. The research bet is that a claim's sensitivity to an evidence swap predicts whether the model will hallucinate on that question when its context conflicts with its memory. We expect this signal to catch failures that text-versus-context checkers miss by construction, and that deleting the evidence cannot isolate.

Core Claim

Our central claim is about claims that are "correct but evidence-insensitive." These are answers a hallucination detector rates as supported, but which do not change when their supporting fact is consistently edited. We claim such answers make up a substantial share of supported RAG answers, and that they are the main source of hallucinations once retrieved context conflicts with what the model memorized.

Detection rests on one quantity. Take a claim $c$ whose supporting value $v$ appears in the retrieved documents. We generate $K$ edited document sets, each replacing every mention of $v$ with an alternative $v_k$ of the same type. Each edit is checked for fluency and for whether it was fully applied. The tracking score is

$$T(c) = \frac{1}{K}\sum_{k=1}^{K} \mathbb{1}\left[\text{rerun answer asserts } v_k\right],$$

that is, the fraction of edits in which the regenerated claim adopts the edited value. A second score, $T_{\text{draft}}(c)$, is defined the same way on the reasoning step that quotes or paraphrases $v$. With both scores, each claim falls into one of four cases. It is tracked when both the draft and the answer follow the edit. It shows a draft–answer disconnect when the draft follows the edit but the answer does not. It shows parametric override when neither follows. It is spurious when the answer changes but not in the direction of the edit.

Two design choices separate this from earlier perturbation tests. First, the test checks that the answer moves to the edited value, not merely that it changes. Second, every mention of the value is edited at once. Together these address the objection that natural-language redundancy blurs counterfactual flips in factual text, raised by FaithCoT-Bench.

How This Sits Against Prior Work

Several papers motivate the gap. Correctness is not Faithfulness in RAG Attributions inserted answer snippets into other documents and found up to 57% of Command-R+ citations were unfaithful. It is diagnostic only, uses one model and one benchmark, and tests whether the citation moves rather than whether the answer moves. We reverse the direction of intervention: we edit the evidence and watch the claim. The thinking-draft work, Measuring the Faithfulness of Thinking Drafts in Large Reasoning Models and RFEval, shows that a reasoning model's final answer often ignores its own reasoning. Draft-to-answer consistency falls below 30% on hard tasks, and 49.7% of traces are unfaithful across 12 models. Both studies work on closed-book math and logic and inject errors into the reasoning itself. We bring the same split, between the document-to-draft link and the draft-to-answer link, into open-book RAG. There the thing intervened on is the evidence, so no reasoning prefix has to be forced.

Other papers supply methods we reuse or compete with. SelfCite scores a citation by how much the response's probability drops when the cited sentence is removed. This is the closest existing causal test, and it is our main ablation contrast. CF-RAG and RC-RAG apply counterfactuals in RAG to the query or the instruction, and use answer agreement to improve accuracy or decide when to abstain. Neither tests whether a particular claim depends on a particular piece of evidence, and neither examines reasoning traces.

Mechanistic work frames the questions our behavioral test can help decide. ReDeEP attributes RAG hallucinations to feed-forward layers that inject memorized knowledge and override attention heads that copy from context. FACTUM finds instead that stronger parametric activity accompanies correct citations. The mechanistic RAG study in Quantifying reliance on external information over parametric knowledge during Retrieval Augmented Generation (RAG) using mechanistic analysis finds that context shuts down memory recall by 10–35x on clean single-hop prompts. These accounts disagree. Our four-way classification shows which regime a given claim falls into, on any model including closed ones. We make no mechanistic claim; we only report the behavior.

Research Program

The pipeline has three parts. The first is claim extraction. It splits a RAG answer, and its reasoning trace when the model produces one, into atomic claims. For each claim it finds the supporting value spans in the retrieved documents, using the model's citations where they exist and a span-matching pass otherwise. A claim with no supporting span is labeled baseless outright. This corresponds to RAGTruth's "introduction of baseless info" category in RAGTruth, which that paper finds is the most common hallucination type.

The second part is edit construction. An LLM proposes type-matched alternative values, such as dates for dates and entities of the same class. These are substituted at every coreferent mention across all retrieved documents. A separate verifier checks that the original value no longer appears and that the edited documents still read coherently. Failed edits are discarded and counted, because the edit-failure rate is a cost we must report.

The third part is rerun and scoring. The model is rerun on the same question with the edited documents, using $K=3$ to $5$ edits. We compute $T$ and $T_{\text{draft}}$ and assign the four-way label. The cost is $K$ extra generations per audited answer. We compare against RC-RAG's three passes and report quality as a function of $K$.

The study has two parts. The descriptive part measures how common each label is across models and tasks, and whether reasoning models and models trained with outcome-based reinforcement learning show more draft–answer disconnects in RAG, as their closed-book behavior would predict. The predictive part tests whether $T$, computed on the original unedited run, predicts hallucination on a separately built conflict test. If the auditor predicts well, we then use $T$ as a preference signal, much as SelfCite uses probability drop. We treat this training step as conditional, not as a headline contribution.

Baselines

The naive reference point is to trust every claim that a text-versus-context check rates as supported. We also run a closed-book comparison: answer without retrieval, and flag a claim as possibly memorized if the closed-book answer agrees. This costs one extra pass, and it is the cheapest heuristic that is still causal in spirit.

The strongest existing detectors come from two lines of work. The first is the fine-tuned RAGTruth critic, Llama-2-13B, which reports 78.7% response-level F1 and 52.7% span-level F1 on RAGTruth in RAGTruth. It is strongest at spotting unsupported text. The second is ReDeEP, which reports AUC 0.7458 on RAGTruth with LLaMA2-7B, against RAGAS at 0.7290 and SelfCheckGPT at 0.5844 ReDeEP. It is the strongest white-box detector of the "memory overrides context" failure our score also targets. We run ReDeEP on LLaMA2-7B for direct comparability, reproducing its reported AUC first, and the RAGTruth critic as released. RLSeek RLSeek is included if available. The file reports no raw number for RLSeek on NewsSum, so we compare to it only on our own runs.

On RAGTruth, our method counts as beating these baselines if it matches their AUC. Our main claim is about the conflict test, where the bar is a clear AUC margin over all of them under the same generator and the same retrieved documents.

The key ablation replaces evidence editing with evidence deletion. This is SelfCite's probability-drop score, plus a black-box variant that compares regenerated answers with and without the cited sentence. Both get the same number of extra passes. The ablation isolates our main ingredient: replacing the evidence rather than removing it. A second ablation edits only the cited mention, not every mention, to measure how much consistent editing matters. A third uses $T$ alone versus $T$ combined with $T_{\text{draft}}$, to test whether looking at the reasoning trace adds any predictive power. We also include RC-RAG, with its reported 14.77% carefulness gain on RC-TQ and RC-NQ RC-RAG, rerun at equal pass budget on our splits.

Why not just…?

Why not just delete the evidence and see if the answer changes, as SelfCite does? When the model's memory agrees with the document, deleting the document leaves the answer intact whether or not the model used it. Deletion therefore cannot separate "used the evidence" from "memory happened to agree," and that is exactly the correct-but-insensitive case we target. Deletion also fails when other documents repeat the same fact. Replacing the value everywhere avoids both problems. This is still the most serious threat to the proposal, because deletion is cheaper and needs no edit generator. The deletion ablation settles it directly. If deletion matches our conflict-test AUC, we will report that the edit step is unnecessary.

Why not just use a fine-tuned hallucination detector or an LLM judge? These read the answer against the context. A correct, supported answer passes them no matter how it was produced. Correctness is not Faithfulness in RAG Attributions shows judges accept such citations even when 57% are unfaithful. These detectors should stay strong at catching hallucinations already visible in the text. The experiment checks whether they lose ground specifically on the conflict test, where the failure only appears under changed evidence.

Why not just use white-box signals such as ReDeEP or FACTUM? They need hidden states and per-model calibration. They also disagree about whether parametric activity is harmful: ReDeEP treats it as the cause of hallucination, while FACTUM finds it accompanies correct citations. Head-to-head on LLaMA2-7B shows whether behavioral sensitivity carries information the internals do not. If ReDeEP matches us there, the case for our method narrows to closed models, where white-box methods cannot run.

Why not just compare against a closed-book answer? Agreement with the closed-book answer shows memory could have produced the answer, not that it did. We expect this baseline to over-flag, and it is included as the naive causal reference.

Proposed Experiments and Decisive Checks

We need a conflict test that is independent of the probe. From NQ/KILT and HotpotQA questions, we build test contexts in which the gold answer-bearing value is edited consistently to a value never used by the auditor. The new value becomes the correct answer, and a hallucination is any answer asserting the original value or some other unsupported one.

Generators are LLaMA2-7B (for ReDeEP comparability), Llama-3.1-8B, a reasoning model with visible thinking drafts in the Qwen3 family, and one closed API model. We report detection AUC on the conflict test for $T$ and every baseline. We also report standard AUC and F1 on RAGTruth, and the four-way label distribution per model. The reasoning-trace analysis tests whether draft–answer disconnects in RAG are common in reasoning models, as the closed-book findings in Measuring the Faithfulness of Thinking Drafts in Large Reasoning Models would predict.

The decisive check is a specific subset: answers the RAGTruth critic rates as supported but that have low $T$. If these hallucinate on the conflict test at a much higher rate than high-$T$ supported answers, the core claim holds. If the two groups hallucinate at similar rates, evidence sensitivity adds nothing beyond text-level checking, and the claim fails.

Two secondary checks follow. First, $T$ must beat SelfCite-style deletion at equal pass count. Second, edit-failure rates must stay low enough that the auditor covers most claims. We report coverage alongside every AUC, so that abstaining from hard edits cannot inflate the results.