When you ask a Large Language Model (LLM) a question, it draws from two distinct sources of information: the "parametric memory" it acquired during pre-training and the "contextual information" provided in your prompt. Ideally, these sources work in harmony. However, when they disagree—such as when a Retrieval-Augmented Generation (RAG) system provides a document that contradicts the model's internal weights—a "tug-of-war" ensues.
Recent research identifies a persistent "stubbornness" in LLMs, where they frequently prioritize outdated or incorrect internal facts over explicit evidence provided in the context.
The Scaling Paradox of Model Stubbornness
Intuitively, one might expect larger, more capable models to be better at following instructions and incorporating new information. However, Shayne Longpre and Ankit Singh Rawat have demonstrated an inverse scaling effect: as models grow in size, they often become more resistant to contextual updates.
This happens because larger models develop stronger, more confident internal representations of facts during pre-training. Ankit Singh Rawat describes this as a "controllable working memory" problem, where the model's internal prior becomes so "stiff" that it overrides the inference-time context window. This "cognitive inertia," as framed by Xinxin You, means that the more a model "knows," the harder it is to convince it otherwise.
Why Models Resist: Memory Strength and Popularity
The intensity of this conflict is not uniform across all topics. Jian Xie found that a model's resistance is often proportional to the "popularity" of a fact in its training data. Highly frequent facts create such entrenched beliefs that they become nearly impossible to override.
Yuepei Li links this behavior to memory strength—the consistency with which a model answers paraphrased versions of the same question. When memory strength is high, the model acts like a "stubborn sloth," ignoring the prompt. Conversely, Elena Stringli shows that this rigidity extends to the "redefinition task," where larger models struggle to accept counterfactual rules (like "redefine $\pi$ as 100") because they are anchored to their original training data.
Measuring the Tug-of-War
To quantify this phenomenon, researchers have developed information-theoretic metrics:
- Persuasion and Susceptibility Scores: Kevin Du introduced these to measure how easily a model's internal beliefs are swayed by context.
- Fisher Susceptibility: Tianyu Liu developed a method to measure this 70x faster than previous sampling methods, confirming that instruction tuning actually makes models more sensitive to context, even if their underlying priors remain strong.
Strategies for Contextual Faithfulness
Researchers are exploring three primary ways to force models to "trust the evidence" over their memory.
1. Context-Aware Decoding
Instead of changing the model's weights, we can change how it chooses the next token. Weijia Shi introduced Context-aware Decoding (CAD), which contrasts the model's output distribution with and without the context. By amplifying the difference, CAD effectively "mutes" the internal prior and forces the model to focus on the provided text.
2. Knowledge-Aware Alignment
Standard fine-tuning often fails to address stubbornness. Baolong Bi proposed Context-DPO, which uses Direct Preference Optimization to specifically train models to prefer context-faithful responses over those based on internal memory. Similarly, Ankit Singh Rawat introduced Knowledge-Aware Finetuning (KAFT) to teach models to remain robust against irrelevant noise while prioritizing relevant context.
3. Internal Interventions
Some methods look "under the hood" to resolve the conflict. Pengcheng Huang developed ParamMute, which identifies specific Feed-Forward Network (FFN) layers responsible for storing factual knowledge and strategically "mutes" them during RAG tasks. This forces the model's attention mechanism to rely more heavily on the external context.
The Path Forward
While scaling models improves their general reasoning, it paradoxically makes them more difficult to steer when information changes. As Isabelle Augenstein notes in From Internal Conflict to Contextual Adaptation of Language Models, resolving these "intra-memory conflicts" is essential for building reliable systems that can adapt to a rapidly evolving world.