Retrieval-augmented generation (RAG) pastes documents fetched by a search system into a language model's prompt so it can answer from them. Two widely cited studies report a strange result: padding that prompt with completely unrelated random documents raises accuracy by roughly 30%, while adding topically similar but wrong documents ("distractors") causes steep drops. The explanation offered is that random text spreads the model's attention more evenly and keeps it from locking onto a misleading passage. Both studies, however, ran on Natural Questions, a Wikipedia-based benchmark that large models have almost certainly seen during pretraining. That leaves a simpler explanation untested. Random padding may just drown out a weak or misleading retrieved context, so the model falls back on answers it memorized from the benchmark's own source text.
This note proposes a controlled test that separates these two explanations. We hold each question and its documents fixed and change only whether the model's memory can help. Entities are renamed to fictional ones so that memory is useless, or the answer is swapped so that memory actively misleads. We then measure how random noise and distractors change accuracy, what gets copied into wrong answers, and which internal pathways carry the answer. If the benefit of noise vanishes when memory is taken away, then a published robustness finding is partly a contamination artifact. It would also mean that noise-filtering methods do not throw away a useful regularizer, contrary to what the literature currently suggests.
Core Claim
The claim is that the accuracy change caused by adding context noise splits into two parts. One is a context-reading effect, which acts on how the model extracts answers from the prompt. The other is a parametric fallback effect, in which noise weakens the model's reliance on the prompt and lets memorized answers through. The second part is large only on questions the model has memorized. We predict that on answer-renamed ("fictional") versions of Natural Questions items, random documents give little or no gain. On original items the gain reported in Rethinking Relevance and The Power of Noise should reappear, concentrated on items the model answers correctly with no context at all. Distractor damage should behave the other way. It should be largest on fictional items, where no memory counterweight exists, and its errors should mostly be copies of entities that appear in the distractor. That pattern is the surface-overlap shortcut described in the reading-comprehension literature (Sugawara et al.; Inui et al.), now showing up at retrieval time.
This connects the four directions in specific roles. RAG is the setting. Context noise is the thing we manipulate. Contamination is the hidden confound we control. Shortcut exploitation is the error mechanism we diagnose, through where wrong answers come from.
Relation to Prior Work
Several papers motivate this proposal. The two noise studies (Rethinking Relevance; The Power of Noise) separate gold, distractor and random documents on Natural Questions. They report Llama 2 7B falling from 56.42% with the gold passage to 37.81% with 18 distractors, and rising from 18.66% to 25.30% when 15 random documents are added to 4 retrieved ones. They do not control for pretraining exposure. We rerun their protocol with exposure as the manipulated variable.
CofCA rewrites Wikipedia passages with fictional entities. It shows GPT-4 exact match falling from about 70% to about 53% on 2-hop counterfactual items, which means memorized facts inflate benchmark scores. CofCA supplies our rewriting technique, but it gives only the target passage and never varies context noise. Its data is also static, so it will leak into future training sets. We apply renaming per experiment, so fresh fictional variants can be generated for each new model.
Two mechanistic papers (Quantifying reliance on external information; From RAGs to rich parameters) use causal tracing. This technique restores intermediate activations to measure how much each layer carries the answer. They show that when a clean context is present, the model mutes its feed-forward (MLP) memory pathway. In Llama-2 the causal effect at the last subject token falls from 0.2 to 0.0375. They studied only clean, single-mention contexts on models of 7B parameters or fewer, and name noisy contexts as an open question. We reuse their method to test whether noise switches the memory pathway back on.
The gullibility work constrains the design. Why So Gullible? shows that models trust counterfactual retrieved text over correct memory, with FiD falling from about 63% to about 30%. That argues against assuming memory always wins, which is why we add an answer-swapped condition where memory and context conflict. Counterfactual Reasoning for RAG names the "correlation trap", in which dense retrievers surface high-overlap wrong passages. Neither paper examines contamination.
Filtering methods are what our result would reinterpret. The information-bottleneck filter (IB-RAG) reaches 21.5 exact match against 14.7 for unfiltered top-5 retrieval and 16.2 without retrieval. Ext2Gen raises Llama3.1-8B from 0.473 to 0.557. Both strip context heavily. The noise-paradox literature implies they discard helpful noise, and our experiment tests whether that implication holds once memory is controlled. On scale, RARE and Symbolic Perturbations find that larger models are sometimes less robust. The fallback hypothesis offers one explanation: larger models memorize more, so their behavior under noise leans more on memory.
Research Program
We build three matched versions of each question from Natural Questions and the multi-hop sets HotpotQA and MuSiQue.
The original version is left untouched, so memory can help.
The fictional version renames the question's subject and answer entities consistently across the question, gold passage and distractors. Replacement names are matched in token length and corpus frequency. The frequency matching addresses the finding in Snoopy that frequency drives few-shot accuracy.
The swapped version keeps the real subject but replaces the answer with a plausible real entity of the same type. Memory then conflicts with the context.
Random documents are left untouched in all three versions. For each version we build context sets that follow the gold, distractor and random composition used by the two noise studies: gold only, gold plus k distractors, realistic top-4 retrieval, and realistic retrieval plus k random documents, with k from 0 to 18. Distractors are fetched with both Contriever and BM25.
Each item also gets a memorization score. This combines closed-book correctness with the likelihood the model assigns to the true answer string given the question. It is a graded proxy for exposure in the spirit of Nicolas Yax et al.. Every wrong answer is labeled by source. It may be copied from a distractor, match the model's closed-book answer, match the original pre-rename answer (a telltale leak on fictional items), or be other.
Mechanistically, we run causal tracing and attention knockout, as in the Knowns-based studies, on Llama-2-7B and Llama-3.1-8B across the context conditions. We ask whether random padding raises the causal effect of the memory pathway on original items but not on fictional ones. We also record attention entropy over context tokens, so the regularization explanation gets a fair direct test. Behavioral experiments extend to a larger model, Qwen3-32B or a 70B-class Llama, to test the scale interaction.
Baselines
The naive reference point is closed-book answering with no retrieval, alongside plain top-4 retrieval with no noise manipulation. These anchor how much memory alone gives on each version. By construction, closed-book accuracy on fictional items should be near zero, and any remaining accuracy reveals renaming leaks.
The key comparison is a replication of the random-noise injection protocol from Rethinking Relevance and The Power of Noise on the original version. We use the same Llama 2 7B, retriever and document counts, and aim to reproduce their reported gain from 18.66% to 25.30%. Replication is the precondition for everything else. If we cannot reproduce it, the proposal shifts to explaining why. Our result "beats" this account if, on fictional items, the random-noise gain falls below half its original-version size, with a bootstrap confidence interval that excludes the original gain.
The strongest existing method competes on the practical question of whether to keep noise or filter it. That method is the information-bottleneck filter from IB-RAG, which reports 21.5 exact match on Natural Questions against 14.7 for unfiltered retrieval. Ext2Gen serves as a second filtering baseline, with 0.557 on Llama3.1-8B. We run the released filters with their published settings on all three versions using the same generator and retrieved pool. We do not retune either filter or our noise conditions beyond a shared small grid over k. If our hypothesis is right, filtering should beat random padding by a wider margin on fictional items than on original items. That would show that the apparent value of noise disappears once memory cannot help.
The prompt baseline adds the instruction "if the context does not contain the answer, say unanswerable". Improving Zero-shot Reader found this kind of instruction contributes to large gains, and UAEval4RAG reports that restrictive rejection instructions improve unanswerable-query handling by nearly 80%. We test whether the instruction interacts with the fallback effect.
The ablation that isolates our key ingredient is renaming applied without any noise manipulation. It measures how much of the original-versus-fictional gap exists in clean conditions, so that the noise-by-memory interaction is not confused with a plain difficulty shift. It is paired with a noise-only ablation on original items.
Why not just…?
Why not just split Natural Questions by closed-book correctness instead of rewriting items? This is the real threat, and it is cheap. We run it as a first analysis. If the random-noise gain is confined to items the model answers correctly closed-book, that is already evidence for fallback. The weakness is confounding. Items a model knows also tend to be about popular entities, with clearer passages and better retrieval, so a stratified gap may reflect item difficulty rather than memory. Renaming keeps the question and passages fixed and changes only whether memory applies. If stratification and renaming give the same decomposition, we will report that the cheap method is enough, which is itself a useful result.
Why not just evaluate on post-cutoff data such as RARE-Set or CofCA? RARE uses 2020–2025 financial and policy documents, where topic, style and difficulty all differ from Natural Questions. A different noise effect there could come from any of these differences. We do use RARE-Set as a validity check: the fallback account predicts that random padding there shows little gain. CofCA gives only the target passage, so its authors never tested noise composition. We reuse its rewriting approach but add the manipulation it lacks.
Why not just measure attention entropy directly and accept the regularization story? We do measure it. The two explanations make different predictions about the source of correct answers. Regularization predicts that the extra correct answers come from the context in every version. Fallback predicts that on original items they match closed-book answers, and that in the swapped version random padding increases the rate of answers that follow memory against the context. Entropy that rises equally across versions while gains appear only on original items would count against pure regularization.
Why not just switch to BM25, which The Power of Noise found often beats dense retrieval? BM25 changes which distractors appear, not whether memory is present, so it is a different lever. We run both retrievers. The fallback hypothesis predicts that the noise-by-memory interaction holds under both.
Decisive Checks
Three outcomes would settle the question.
The first is the random-noise gain, measured separately on fictional and original items for each model. If the gain on fictional items stays within the confidence interval of the original-version gain, the fallback account fails and the regularization explanation stands on firmer ground.
The second is the swapped version. Here fallback predicts that random padding raises the share of answers that follow memory against the context, while regularization predicts no change.
The third is causal tracing. Fallback predicts that random padding restores the memory pathway's effect toward its closed-book level of about 0.2 on original items, while the effect stays near its clean-context level on fictional items.
We also check the scale interaction. If larger models show larger fallback effects, part of the robustness contradictions reported in RARE could reflect memorization rather than reasoning. Whatever the result, the renaming harness is reusable. It gives every future RAG noise benchmark a way to report how much of its effect depends on the model having seen the test.
References
- Rethinking Relevance: How Noise and Distractors Impact Retrieval-Augmented Generation
- The Power of Noise: Redefining Retrieval for RAG Systems
- Benchmarking Machine Reading Comprehension: A Psychological Perspective
- What Makes Reading Comprehension Questions Easier?
- CofCA: A STEP-WISE Counterfactual Multi-hop QA benchmark
- Quantifying reliance on external information over parametric knowledge during Retrieval Augmented Generation (RAG) using mechanistic analysis
- From RAGs to rich parameters: Probing how language models utilize external knowledge over parametric information for factual queries
- Why So Gullible? Enhancing the Robustness of Retrieval-Augmented Models against Counterfactual Noise
- Counterfactual Reasoning for Retrieval-Augmented Generation
- An Information Bottleneck Perspective for Effective Noise Filtering on Retrieval-Augmented Generation
- Aligning Extraction and Generation for Robust Retrieval-Augmented Generation
- RARE: Retrieval-Aware Robustness Evaluation for Retrieval-Augmented Generation Systems
- Evaluating RAG Robustness to Symbolic Perturbations
- Snoopy: An Online Interface for Exploring the Effect of Pretraining Term Frequencies on Few-Shot LM Performance
- Studying and improving reasoning in humans and machines
- Improving Zero-shot Reader by Reducing Distractions from Irrelevant Documents in Open-Domain Question Answering
- Unanswerability Evaluation for Retrieval Augmented Generation