Cited but Unused: Parametric Provenance in Retrieval-Augmented Generation

Tanzeel Abbas, Tehseen Zia, David WindridgearXiv 2026 · arXiv

We rely on Retrieval-Augmented Generation (RAG) to keep Large Language Models (LLMs) grounded in facts. When a model provides a citation, we assume that’s where the information came from. But what if the model is just "post-rationalizing"—answering from its own memory and then finding a passage that happens to agree?

A new paper, "Cited but Unused: Parametric Provenance in Retrieval-Augmented Generation," explores this deceptive behavior. The authors find that as models get larger and more "knowledgeable," they are actually more likely to ignore their retrieved context in favor of their internal weights, all while confidently attaching a citation to the ignored source.

TL;DR

LLMs often cite sources they didn't actually use to generate an answer, a phenomenon called "post-rationalization." By using counterfactual edits to retrieved passages, the authors show that 1.2% to 15.7% of cited claims come from internal memory rather than context, and standard verification tools are completely unable to detect this failure.

What Problem Does This Paper Address?

In RAG, we distinguish between support (does the passage agree with the claim?) and provenance (did the passage produce the claim?).

Current evaluation methods focus almost entirely on support. If a model says "The Eiffel Tower was completed in 1889 [1]" and passage [1] says the same, we mark it as a success. However, if the model already "knew" that date and would have said it regardless of what the passage contained, the citation is a lie of provenance. This matters because if the passage had been wrong or updated, a model relying on its own memory would have ignored the new evidence, potentially spreading outdated or conflicting information while appearing grounded.

Methods: The Counterfactual Test

To measure this without guessing at the model's internal states, the authors proposed counterfactual value substitution.

  1. Identify a value: Find a specific fact (like a year) that appears in both the model’s answer and the retrieved passages.
  2. Intervene: Replace that value in the retrieved passages (including titles) with a false variant (e.g., changing 1889 to 2029).
  3. Re-query: Ask the model the question again with the edited context.
  4. If the model adopts the false value, it used the context.
  5. If the model repeats the original (true) value, it used its parametric memory.

The authors tested this across the Qwen2.5 family (0.5B to 14B parameters) and Llama-3.1-8B to see how model size and family affect this behavior.

Datasets

The study primarily used ASQA (factoid questions requiring long-form answers) paired with GTR retrieval. They also validated the phenomenon on ELI5 (long-form explanations) and QAMPARI (questions with multiple answers) to ensure the results weren't specific to one type of query.

Key Insights and Ideas

  • Scaling exacerbates the problem: Within a model family, larger models are more likely to rely on their internal memory. As models get "smarter," they become more stubborn.
  • Family matters as much as size: Llama-3.1-8B showed a parametric provenance rate of 15.7%, nearly double that of the similarly sized Qwen2.5-7B (8.2%). This suggests that training recipes significantly impact how much a model trusts its context vs. its weights.
Figure 1: Parametric provenance rate against parameter count. Bars are Wilson 95% intervals. Llama-3.1-8B sits above every Qwen model including one twice its size, indicating that family matters independently of scale.
  • Citations are "cheap": Models answering from memory attach citations just as frequently as those answering from context (often >90% of the time). A citation marker is no guarantee of context use.
  • Standard NLI is blind: Natural Language Inference (NLI) tools, the industry standard for RAG verification, perform no better than chance at detecting provenance failures. Because the model picks a citation that matches its output, the NLI check passes even when the provenance is wrong.

Key Results

  • Prevalence: Parametric provenance affects between 1.2% and 15.7% of citable claims.
  • Detection: The authors compared ten attribution methods. They found that token-level contrast—comparing the model's probability distribution for a token with and without the context present—is the strongest signal for detecting these failures (mean AUROC of 0.79).
Figure 2: Attribution methods grouped in terms of what they measure. Each plot point is one model; the bar is the group mean. Methods hence cluster by quantity rather than by lineage.
  • The "Oracle" Gap: While token-level contrast is a strong signal, it requires knowing exactly which token to check. When a detector has to scan an entire answer without knowing where the "memory-based" claim might be, performance drops significantly.

Limitations and Future Work

The authors highlight several hurdles for future research:

  1. Closed Models: The best detection methods require token-level log-probabilities, which are often hidden by commercial API providers (like OpenAI or Anthropic).
  2. Deployability: Even the best detector isn't yet accurate enough to automatically "fix" RAG outputs. At current base rates, flagging a claim as "parametric" results in too many false positives to be useful for automated repair.
  3. The "Knowledge vs. Use" Problem: The signal they found measures what the model knew before the context arrived. Separating what a model knew from what it actually used during generation remains an open challenge.

For engineers building RAG systems, this paper is a sobering reminder: just because a model provides a citation that checks out doesn't mean it actually read the manual.

Summary updated

Abstract

Retrieval-augmented generation systems attach citations to attribute claims to their sources. These citations frequently misstate provenance, and standard verification cannot detect this. We measure provenance by counterfactual value substitution: a fact in the retrieved passages is replaced with a false variant and the model re-queried. We find that across five sizes of one model family and a second comparator family, 1.2% to 15.7% of citable claims derive from parametric memory rather than the retrieved context. At the most conservative substitution distance these figures are 4.6% and 8.5% on the two models we tested at it, with the rate rising with model size within a family. We further find that natural language inference verification, the standard approach, gives results indistinguishable from chance across models, even when restricted to the cited passage. We determine that the cause is structural: a model answering from memory selects a citation that agrees with its output. Comparing ten attribution methods on identical claims, the contrast between the model's distribution with and without context best separates provenance. Acting on that signal does not improve provenance honesty at these base rates. The resulting detector is hence still not accurate enough to act on: it measures what the model knew rather than what it used, and separating those remains the open problem.
BibTeX
@misc{abbas2026cited,
  author = {Tanzeel Abbas and Tehseen Zia and David Windridge},
  title = {{Cited but Unused: Parametric Provenance in Retrieval-Augmented Generation}},
  year = {2026},
  eprint = {2611.17161},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.17161},
}
Figure