Large Language Models (LLMs) simultaneously operate under two distinct paradigms: they act as static encyclopedias, drawing on facts baked directly into their weights during pre-training (parametric knowledge), and as dynamic in-context processors, reading and reasoning over information provided directly within a prompt (contextual knowledge). As Du describes, this dual nature creates a fundamental tension when these two sources disagree—such as when a Retrieval-Augmented Generation (RAG) system injects a document containing updated or counterfactual facts. Understanding how a model adjudicates these internal contradictions is critical for building reliable, factually accurate systems.
The Internal Mechanics of Conflict
What happens inside a Transformer network when a prompt asserts something that contradicts its training data? Mechanistic analyses reveal that the model's internal layers detect this friction long before it generates a single token.
According to findings in Analysing the Residual Stream of Language Models Under Knowledge Conflicts, a conflict manifests as distinct activation patterns within the residual stream—the internal data highway of the Transformer. This friction triggers what Ortu et al. describe as a "competition of mechanisms." The factual prior and the novel context are processed along separate internal pathways at different token positions, eventually converging in the final layers where the model must choose which signal to surface.
To locate the exact arbitration points of this competition, Jin et al. isolated specific attention heads within the Transformer architecture. They identified two competing classes of circuits:
- Memory Heads: Attention heads that actively pull forward the model's pre-trained parametric facts.
- Context Heads: Attention heads that prioritize attending to the provided prompt text.
By surgically pruning or dampening these specific heads, the authors demonstrated that it is possible to shift the model's behavior along the spectrum of memory reliance versus context adherence without any retraining.
[Input Prompt with Conflict]
│
▼
[Residual Stream Detection] ──► Triggers Competition
│
├──► Memory Heads ──► Pulls Parametric Prior (Internal Memory)
│ │
│ ▼
├──► Context Heads ──► Pulls Contextual Evidence (External Prompt)
│ │
▼ ▼
[Final Layers Arbitration] ──► Output Generation
Failure Modes: Stubbornness vs. Gullibility
When the internal arbitration between these mechanisms fails, models typically fall into one of two problematic behaviors:
Stubbornness (Entity Bias)
Models often exhibit an intense stubbornness, ignoring explicit contextual evidence in favor of their pre-trained weights. As Wang et al. and Song et al. observe, this entity bias is highly correlated with pre-training frequency. If a model encountered an entity thousands of times during training, its parametric prior is incredibly strong, causing it to reject updated or niche information provided in the context window.
Gullibility (Counterfactual Vulnerability)
Conversely, models can be overly gullible. In high-noise environments, a model might abandon its correct internal knowledge base to follow erroneous, outdated, or malicious external context (Hong et al.). Without a mechanism to evaluate the reliability of external inputs, the model experiences a "factual flip," where a previously accurate model becomes incorrect due to poor retrieval data.
Strategies for Resolution
To prevent models from blindly defaulting to internal biases or falling prey to noisy context, several frameworks intervene at different stages of the generation pipeline.
Structured Inference
Rather than feeding raw retrieved text directly into the generation loop, frameworks like Chain-of-Note (Zhang et al.) introduce an intermediate reasoning step. The model is forced to generate sequential "reading notes" that explicitly evaluate the relevance and trustworthiness of each retrieved document before formulating a final answer. This structured evaluation helps the model filter out counterfactual noise while leveraging its internal knowledge when retrieval fails.
Decoding-Time Interventions
Instead of altering model weights, Adaptive Contrastive Decoding (Kim et al.) dynamically calibrates the influence of external text during generation. By measuring entropy-based uncertainty metrics at each token step, the system determines whether the context is providing high-quality information or confusing noise, adjusting the decoding probabilities to protect the model from misleading external claims.
Context-Priority Alignment
To systematically enforce grounding, researchers have turned to alignment techniques. Zhang et al. and Song et al. utilize reinforcement learning to reward models for generating answers strictly consistent with provided context, while penalizing responses that rely on contradictory internal memory. This training incentivizes the model to prioritize external evidence even when it directly challenges its pre-trained parameters.