Large Language Models (LLMs) are not blank slates. During pre-training, they acquire a vast "parametric memory"—a compressed representation of the world derived from their training data. However, when we use these models in Retrieval-Augmented Generation (RAG) or In-Context Learning (ICL), we ask them to prioritize a different source: the "contextual knowledge" provided in the prompt.
A growing body of research reveals a fundamental "tug-of-war" between these two knowledge sources. When the model’s internal memory contradicts the provided text, the model often defaults to its pre-trained biases, leading to a phenomenon known as parametric knowledge bias or contextual unfaithfulness.
The Internal Tie-Breaker Mechanism
The core challenge is that LLMs do not treat external context as an absolute authority. Instead, they weigh it against their internal priors. Hung-Ting Chen, Eunsol Choi, and Michael J. Q. Zhang found that when external evidence is split or ambiguous, models use their internal parametric knowledge as a "tie-breaker" to resolve the conflict. Crucially, they do this without signaling any uncertainty, presenting a biased answer with the same confidence as a grounded one.
This bias is not just a failure to read; it is a systematic preference. Kai Zhang and colleagues describe this behavior as the "Stubborn Sloth" effect, where models resist updating their beliefs even when presented with clear, contradictory evidence. This is particularly visible in KnowShiftQA, a benchmark by Tianshi Zheng and others, which shows that RAG systems suffer a 22–27% performance drop when textbook knowledge shifts away from what the model learned during pre-training.
Why Models Ignore the Context
The failure to stay faithful to context often stems from how the model's internal weights act as a filter. Alexander Rubinstein and Seong Joon Oh demonstrated that if a model’s incorrect internal answer appears anywhere as "noise" within a retrieved document, the model is significantly more likely to latch onto that noise and ignore the correct evidence provided in the same document.
This suggests that hallucinations are not always stochastic errors. As Sekh Mainul Islam and Syed Islam argue in their work on subspace disentanglement, hallucinations are often a result of systematic internal alignment with parametric memory. In the model's hidden states, the signal from the pre-trained weights physically overrides the signal from the input tokens.
This conflict extends to evaluation. Fan Bai and Mark Dredze found that LLMs used as evaluators exhibit a similar bias, penalizing responses that are contextually accurate but contradict the evaluator's own internal (and potentially outdated) knowledge.
Measuring and Mitigating the Conflict
To address this, researchers have developed benchmarks that force models into "counterfactual" scenarios where they cannot rely on memory. FaithEval, introduced by Yifei Ming, tests if a model can stay faithful to context even when told "The Moon is Made of Marshmallows." Similarly, CofCA by Jian Wu uses "fake facts" to prove that LLMs struggle with multi-hop reasoning when their memorized shortcuts are removed.
Several strategies have emerged to recalibrate this balance:
- Contrastive Decoding: Methods like CLeHe (Zexuan Qiu) and DeCK (Baolong Bi) improve faithfulness by contrasting the model's output distribution with and without the provided context. By amplifying the "delta" provided by the context, these methods help the model overcome the gravity of its internal priors.
- Causal Inference: Jifan Yu and Yifan Xu proposed subtracting the "counterfactual" response (what the model would say without the context) from the actual response to filter out internal parametric noise.
- Faithfulness-Focused Training: Frameworks like CANOE (Haozhe Zhao) use synthetic tasks and reinforcement learning to explicitly reward models for prioritizing provided documents over internal memory.
Ultimately, the goal of this research direction is to transform LLMs from "stubborn sloths" that parrot their training data into "adaptive chameleons" capable of rigorous, grounded reasoning in dynamic environments.