Large Language Models (LLMs) often fail to reason reliably with newly learned facts. While they can easily memorize a statement like "The capital of Andoria is Copperton," they struggle with complex tasks like reverse question answering ("Which country has Copperton as its capital?") or multi-hop reasoning.
This paper argues that this deficiency stems from shortcut learning. Driven by the causal language modeling objective of predicting the next word, LLMs find it easier to statistically associate the strings "Andoria" and "Copperton" rather than forming a true factual association—an abstract, flexible understanding of the relationship. This superficial co-occurrence is sufficient for direct questions but fails in broader reasoning contexts.
Investigating the Knowledge Gap
To understand how models store these different types of knowledge, the researchers employed three main approaches:
- Probing and Ablation: The authors used "Comparison" and "Negation" ratios to determine if a model truly understands a fact or is merely reacting to the presence of specific words. They also performed layer-wise ablation, resetting specific layers to their pretrained state to pinpoint where different types of knowledge reside.
- Referencing Corpus (Implicit Association): Instead of training on direct "Narrative" text (e.g., "A is B"), they created a "Referencing" corpus where relationships are indirect. For instance, stating "City X is colored green" and "The capital of Country Y is colored green" forces the model to link the entities through an intermediate attribute, breaking the direct word co-occurrence shortcut.
- Active Forgetting: The authors proposed a novel training strategy to force deeper learning.
In the Active Forgetting method, the model is first trained on narrative text until the loss drops. Then, the parameters of the upper layers are reset to their original pretrained values. This "clears" the easy co-occurrence shortcuts learned in the middle layers, forcing the model to "unblock" and learn harder factual associations in the lower layers during a subsequent training pass.
Key Insights: Layer Separation and Unblocking
A major finding of the study is the distinct layer separation between superficial and deep knowledge. Co-occurrence statistics are primarily parameterized in the middle layers of the Transformer. In contrast, true factual associations are stored in the lower layers.
This separation leads to an "unblocking" effect. Once a model reduces its training loss by learning a simple co-occurrence shortcut in the middle layers, it stops attempting to learn the more complex factual association. By resetting those middle layers via Active Forgetting, the researchers successfully unblocked the model's ability to learn deeper, more generalizable associations in the lower layers. Furthermore, training on text with implicit associations (the Referencing style) proved significantly more effective for reasoning than training on plain narrative text, regardless of how heavily the narrative text was paraphrased.
Datasets and Key Results
The researchers evaluated these strategies across several datasets:
- Country-city-animals: A synthetic dataset of artificial associations, ensuring the model had no prior knowledge.
- MQuAKE-T & 2WikiMultiHopQA: Real-world datasets used to evaluate single-hop, multi-hop, and complex reading comprehension.
The results demonstrated a massive reasoning gap when using standard training. On the synthetic dataset, a LLaMA 3 8B model trained on standard "Narrative" text achieved 100% on direct QA but only 38.8% on 2-hop reasoning. However, when trained on "Referencing" text, it maintained 100% QA accuracy while its 2-hop reasoning performance skyrocketed to 92.5%.
The Active Forgetting technique also proved highly successful. Applying it to LLaMA 3 8B on narrative text improved 2-hop reasoning from 38.8% to 51.2% on synthetic data, and from 61.3% to 66.2% on the real-world MQuAKE-T dataset. Notably, these findings were consistent across different model families (LLaMA 3 and Gemma) and scales (7B to 70B parameters).
Limitations and Future Work
While promising, the "Referencing" style requires complex rewriting of facts. Future work could explore using LLMs to automatically rewrite plain text into these more "reasoning-friendly" formats. Additionally, the study focused on learning new facts during finetuning; it remains unclear if these same layer-wise dynamics apply during the initial massive pretraining phase of a model. Finally, exploring how these findings generalize to messier, real-world linguistic forms beyond the two specific styles tested will be a crucial next step.
Summary updated