Telling Grounding from Echoing in Retrieval-Augmented Generation

Retrieval-augmented generation (RAG) is a setup where a language model is handed retrieved passages and asked to answer from them. Several hallucination detectors for RAG ask one question: did the answer change when the context was removed or scrambled? If it did, they conclude the model used the context. That test cannot tell two cases apart. In the first, the model read the passage and extracted the right fact. In the second, the model copied words that happened to overlap with the question. Retrievers and re-rankers are known to favour passages that repeat the query's words over passages that contain the answer. So the second case is common, and it is exactly the case in which the copied answer is wrong.

We propose a detector that probes the context with two opposite edits. The first rewrites the passage so its facts stay the same but its words change. The second keeps the words but changes the fact. A model that is truly grounded should give the same answer under the rewording and should follow the changed fact. A model that is echoing words does the reverse: the rewording throws it off, and the fact change goes unnoticed. The difference between these two sensitivities is a hallucination score. It is aimed at the failure that lexical bias in retrieval creates, which current detectors cannot see.

Core Claim

The key work this builds on and argues against is LUMINA. LUMINA detects RAG hallucinations by combining two signals. The first is context utilization: how far the output distribution moves, measured by Maximum Mean Discrepancy (a distance between two distributions), when the retrieved context is replaced with random text. The second is internal-knowledge processing, read from intermediate layers with the logit lens. LUMINA reports over 0.9 AUROC (area under the ROC curve) on HalluRAG.

Our claim is that scrambling the context mixes two kinds of sensitivity: sensitivity to the passage's surface words and sensitivity to its facts. Separating them should catch a class of hallucinations that a scrambling-based score certifies as well grounded.

The motivating evidence comes from two papers that show this class is large. Collapse of Dense Retrievers builds controlled "foil" passages that repeat the query's words while lacking the answer. Dense retrievers choose these foils "more than 90% of the time," and when GPT-4o is given the foil it suffers "34% drop in accuracy" compared with having no context at all. Language Model Re-rankers are Fooled by Lexical Similarities shows the same weakness in neural re-rankers. On the noisy fact-checking set DRUID, BGE improves on BM25 (the classic keyword-matching ranker) by only 10%, against 46% on Natural Questions.

When a foil reaches the generator and the model echoes it, scrambling the foil removes the echoed tokens. The output therefore shifts a lot, and a scrambling-based score reports high context use for a wrong answer.

We borrow one idea from metabolic engineering, as a design lens and not as a mechanism. Work on acetate toxicity found that engineering faster substrate uptake made the cells sicker. The bottleneck was not how much substrate entered the cell but whether the cell could process what it took in. Scrambling-based detectors measure intake: how much the context moves the output. We want to measure processing: whether the content of the context, as opposed to its form, is what moves the output. The same lens suggests one secondary analysis, a dose–response curve. Adding more foil passages may hurt accuracy in the non-monotonic way that substrate inhibition does, and we will test whether each detector's score follows that curve.

Research Program

Given a query $q$, a retrieved context $c$, and the model's answer distribution $p(\cdot \mid q, c)$, we build two perturbed contexts.

The first is a fact-preserving paraphrase $c^{\text{para}}$. It is rewritten to reduce word overlap with $q$ while keeping every relational fact. Fact preservation is checked by bidirectional entailment with a natural language inference model.

The second is a lexicon-preserving fact edit $c^{\text{edit}}$. It swaps the fact relevant to the answer, for example the tail entity of the relation, for another entity of the same type. Every query-overlapping token stays in place, and the edit implies a new correct answer $a^{\text{edit}}$.

The score is

$$ h(q,c) = D\big(p(\cdot \mid q,c),\, p(\cdot \mid q,c^{\text{para}})\big) \;-\; p(a^{\text{edit}} \mid q, c^{\text{edit}}), $$

where $D$ is a divergence between answer distributions. We will use the Jensen–Shannon divergence over sampled answers, which lets the method work with closed models. The second term is the probability that the model tracks the edited fact. A high $h$ means the model is sensitive to wording and insensitive to facts, which is what echoing looks like.

For open-weight models we also compute a layer-wise version. It applies the logit lens to the paraphrase and edit runs. This lets us ask whether the echoing behaviour shows up in intermediate layers before the final answer is produced, which would connect the method directly to LUMINA's internal-knowledge signal.

The engineering effort goes into the perturbations. On the controlled foil set, the edits are exact, because ColDeR is built from Re-DocRED relation triples: we know the head, relation, and tail and can substitute the tail entity directly. On free-form data (RAGTruth, HalluRAG, DRUID), an instruction-tuned model proposes both perturbations. Each one is filtered automatically, by entailment for paraphrases and by a contradiction check plus a token-overlap constraint for edits. A human audit of a few hundred examples per dataset then estimates how often a perturbation is invalid. Because a bad perturbation could create a false signal, we report detector performance both on the full set and on the audited-valid subset.

Baselines

The naive reference point is a detector with no access to context sensitivity. It thresholds the generator's own answer probability, which is its stated confidence. If our score cannot beat this baseline on foil-induced errors, the idea fails at the first step.

The strongest existing method on our exact question is LUMINA (paper). It is reported at over 0.9 AUROC on HalluRAG and is the method our claim targets most directly. We will re-implement it with its scrambled-context Maximum Mean Discrepancy signal and its logit-lens processing-rate signal, run it on the same open-weight generators, and give it the same number of extra forward passes as our method. Both detectors receive a matched per-task budget for threshold calibration, since LUMINA's authors note that its thresholds depend on the task. Beating LUMINA means two things. On the foil-induced test set we expect a clear AUROC gain, roughly five points or more. On HalluRAG we must stay within about one point of LUMINA. No reported result was found on Lacuna for LUMINA on ColDeR foils, DRUID, or RAGTruth, so we will produce those numbers ourselves.

The strongest supervised competitor is the fine-tuned Llama-2-13B critic from RAGTruth. It reports 78.7% response-level F1 and 52.7% span-level F1 on RAGTruth, against 63.4% and 28.3% for prompted GPT-4. It is the strongest competitor because it is trained on human hallucination labels and needs no perturbation. We will reproduce it on the RAGTruth training split and evaluate every method on the same test split, using both F1 levels. Our method is training-free, so on RAGTruth we count as competitive if we match its response-level F1 within a few points. We count as winning if we beat it on the foil-induced set, where it never saw training examples, and on DRUID.

A deliberately simple lexical baseline thresholds two quantities: the word overlap between the answer and the context, and the gold-from-similar separation ($D_S$) of Hagström et al., computed where gold passages are known. This is the baseline most likely to embarrass the proposal, and the next section explains why.

Three ablations isolate the key ingredient. The paraphrase-only arm and the edit-only arm are each scored alone. A third variant uses the edit arm with an edit that also changes words, so it no longer preserves the query's tokens. If the lexicon-preserving edit carries no advantage over this lexicon-changing edit, then separating words from facts is not what drives the result.

Why not just…?

Why not just use LUMINA, which already reaches 0.9 AUROC? That number comes from HalluRAG, where most errors are not produced by lexical foils. Our argument, that echoing a foil produces a large scrambling shift, predicts a specific failure LUMINA has never been tested on. The foil-set experiment settles this. If LUMINA's AUROC on foil-induced errors stays close to its HalluRAG level, our central claim is wrong and we will say so.

Why not just fine-tune a critic on RAGTruth? The fine-tuned critic is strong on in-distribution summarization and QA, but its training labels were never selected for lexical-foil errors. RAGTruth's annotators also did not analyse the lexical properties of the input passages. We expect it to transfer poorly to the foil set and DRUID, but this is an empirical question that the cross-dataset evaluation answers directly. A critic fine-tuned on a small number of foil examples is a further threat. We will add that variant and report how many labelled foils it needs to catch up.

Why not just threshold word overlap or $D_S$? This alternative is a real threat on the foil set, because the foils are built to have high overlap. Overlap cannot distinguish a high-overlap passage that the model handled correctly from one it echoed wrongly. It also needs gold passages to compute $D_S$, which a deployed detector does not have. The decisive comparison uses a subset of examples where overlap is high and the generator still answered correctly. If the overlap heuristic matches our score there, the perturbations add nothing and the simpler method should be preferred.

Why not just fix the retriever? Retriever objectives such as RAC and DKS from Retrieval Augmented Generation with Rich Answer Encoding reduce the retriever's preference for passages that share tokens with the reference answer, and they report a 12.1% relative Recall@1 gain on MS MARCO. Better retrieval lowers how often foils arrive but does not remove them. DRUID-style noisy web data will still contain foils, so a detector is still needed after retrieval. We will run our detector on top of a better retriever to check that the detector's added value does not vanish.

Why not just use a GPT-4-class judge? Mansour et al. show that LLM relevance judges inherit lexical bias, and RAGTruth reports only 28.3% span-level F1 for prompted GPT-4. We include a prompted judge as a baseline with the same per-example call budget.

Proposed Experiments

The first experiment builds a foil-induced hallucination set. We feed ColDeR's 250-pair bias categories (literal, position, brevity, repetition, and answer importance) to open-weight generators, once with the gold passage and once with the foil. We label each answer correct or incorrect against the Re-DocRED triple. All detectors are scored by AUROC on this set. The decisive prediction is that LUMINA's context-utilization signal is high for foil-echoing errors, while our score separates those errors from correct answers.

The second experiment tests transfer. We evaluate on RAGTruth (response- and span-level F1), HalluRAG (AUROC, compared against LUMINA's reported figure), and DRUID claim verification with re-ranked contexts. DRUID tests whether the effect holds where real lexical distractors occur naturally.

The third experiment is the dose–response study on PopQA. We mix k foil passages with one gold passage, for k from 0 to 8, and stratify by entity popularity. Popularity matters because Mallen et al. found that retrieval misleads GPT-3 on popular facts in about 10% of cases while helping on long-tail facts. We measure whether accuracy falls in a non-monotonic, inhibition-like way as foils are added. We also measure which detector's score rises in step with the falling accuracy.

The decisive checks are three. Our score must beat LUMINA on the foil set. It must not lose to LUMINA on HalluRAG. It must beat the overlap heuristic on the subset where overlap is high but the answer is correct. If any of these fails, the separation of words from facts is not adding information beyond what existing detectors or simple overlap already provide.