Abstract
Large Language Models (LLMs) frequently exhibit hallucinations even when the correct information was present in their training data. This phenomenon, termed knowledge overshadowing, occurs when dominant, high-frequency facts suppress the retrieval of related but less prevalent subordinate facts. This paper introduces PHANTOMCIRCUIT, a framework utilizing mechanistic interpretability to trace the formation of these biases during training. By analyzing the model's internal computational circuits, we demonstrate that overshadowing follows a predictable "U-shaped" evolution and can be mitigated by optimizing specific internal nodes responsible for distinguishing between competing knowledge pairs.
1. Introduction
Despite the vast scale of modern training datasets, LLMs often struggle with precision when facts compete for the same conceptual space. For example, a model might correctly know that "Beijing is the capital of China" and "Shanghai is the economic center of China." However, because the association between "Beijing" and "China" is statistically more dominant, the model may incorrectly identify Beijing as the economic center. This is not a simple lack of data, but a structural bias where dominant patterns mask subordinate ones.
While previous research has identified this behavior at inference, the internal dynamics—how these biases are learned and stored—remained opaque. We address this by treating the LLM as a directed acyclic graph and identifying the specific "knowledge circuits" that drive these predictions.
2. The PHANTOMCIRCUIT Framework
To analyze and mitigate overshadowing, we developed PHANTOMCIRCUIT, which operates through four primary stages:
2.1 Circuit Construction
Using Edge Attribution Patching with Integrated Gradients (EAP-IG), we identify the sparse subgraph (circuit) within the LLM responsible for a specific knowledge pair. This allows us to isolate the components (MLP layers and attention heads) that process the relationship between a subject and its competing attributes.
2.2 Dynamic Analysis Metrics
We track the evolution of knowledge through two key metrics:
- Relative Overshadowing Rate (RO): The frequency with which the model prefers the dominant answer over the subordinate one.
- Loss Proportion (LP): The share of total training loss attributed specifically to the subordinate knowledge.
2.3 Circuit-based Recovery and Mitigation
By tuning edge pruning thresholds within the identified circuit, we can isolate "key nodes"—specifically high-attention heads—that distinguish the subordinate fact from the dominant one. Optimizing these circuits allows us to "unmask" the overshadowed knowledge without retraining the entire model.
2.4 Automated Identification
To scale this analysis, we utilize a simplified version of Relative Pointwise Mutual Information (R-PMI). This allows the framework to automatically detect overshadowed tokens in large datasets without requiring manual labeling.
3. The Life Cycle of a Hallucination
Our analysis reveals a distinct "U-shaped" evolution of overshadowing during the training process.
- The Spike: Early in training, the model prioritizes high-frequency, dominant patterns. This causes the $RO$ to spike toward 100%, as the model over-generalizes the dominant fact to all related prompts.
- The Recovery: As the model reaches a plateau in optimizing dominant facts, it begins to allocate capacity to "harder" subordinate facts. This leads to a gradual decline in $RO$ as the model learns to distinguish between the two.
Scaling Laws of Bias
The dynamics of this recovery are heavily influenced by model and data parameters:
- Model Size ($M$): Larger models experience an earlier onset of overshadowing but recover significantly faster.
- Fact Popularity ($P$): Highly popular dominant facts accelerate the initial overshadowing but also lead to a quicker eventual recovery phase.
- Dataset Size ($D$): Interestingly, larger datasets can actually prolong the recovery phase. While they accelerate the onset of overshadowing, the increased data diversity makes it more difficult for the model to master every specific subordinate instance.
4. Mechanistic Insights: The Culprit Heads
Our mechanistic analysis identified specific "high-attention heads" that act as the primary drivers of overshadowing. In models like Pythia-410m, specific heads (e.g., in layer 5) fail to focus on the distinguishing features of a subordinate prompt during the high-overshadowing phase. Recovery occurs precisely when these heads "emerge" and begin correctly channeling information to subsequent layers.
Ablation studies confirmed this: removing just the top 10% of these identified high-attention heads resulted in a catastrophic drop in the model's ability to distinguish between competing facts, confirming their role as the critical nodes of the knowledge circuit.
5. Experimental Results
We validated PHANTOMCIRCUIT across synthetic datasets, fine-tuning tasks with "virtual knowledge" (to ensure no prior exposure), and natural language data using models like Pythia-410m and GPT2-medium.
- Detection: The framework successfully identified the internal "juncture" where the model's internal ranking of the correct subordinate answer finally overtakes the hallucinated dominant one.
- Mitigation: By pruning edges to focus on distinguishing features, we demonstrated the ability to flip a model's incorrect, overshadowed response back to the correct subordinate answer.
6. Conclusion and Future Work
PHANTOMCIRCUIT provides a window into the internal competition of facts within LLMs. By understanding the U-shaped evolution of knowledge and identifying the specific circuits involved, we can move toward models that are more robust against statistical dominance.
Future work will focus on scaling this analysis to massive models like Llama-3 and exploring how overshadowing affects more complex reasoning tasks, such as multi-hop logic and code generation, where the interference between patterns is likely even more pronounced.
Summary updated