TL;DR
Factual retrieval in Large Language Models (LLMs) is not a simple, localized lookup from a single layer. Instead, it is a distributed, non-contiguous process where models utilize sparse "attribute-computation paths" that frequently skip layers. Remarkably, models also possess highly redundant, deeper "backup" paths capable of retrieving the exact same fact when primary paths are disrupted.
What Problem Does This Paper Address?
Where and how do LLMs store facts? The dominant "folk view" in mechanistic interpretability suggests that factual knowledge is localized within specific multi-layer perceptron (MLP) layers acting as key-value memories. However, this view struggles to explain a persistent paradox: why does model editing (altering a fact) often succeed at layers completely different from where standard localization techniques say the fact is stored?
This paper tackles this gap by shifting the focus from where knowledge sits to how it is computed. The authors investigate the precise sequence of layer transformations—which they term the attribute-computation path—that an entity representation must undergo to successfully recall a target attribute (e.g., transforming "Angela Merkel" to elicit "German").
Methods
The authors introduce a novel activation-patching protocol to identify the minimal subset of layers necessary and sufficient for factual retrieval. The method consists of two main phases:
- Locating the Termination Point ($\ell_{attr}$): Using an operation called activation locking, the authors find the earliest layer where freezing the entity representation and copying it directly to all subsequent layers still allows the model to output the correct attribute.
- Iterative Greedy Search (Isolate Protocol): To find the exact layers required before $\ell_{attr}$, the authors introduce counterfactual prompts (e.g., changing "Angela Merkel" to "Bill Gates"). They systematically skip layers and patch in counterfactual activations. If the model still predicts the correct attribute ("German"), it proves the skipped layers were non-essential. This isolates a minimal, irreducible path of required layers.
By relaxing constraints, the protocol can also search for Alternative Minimal Paths—deeper, secondary circuits that the model can recruit if the primary path is blocked.
Datasets
The experiments were conducted using the CounterFact dataset, a widely recognized benchmark for factual knowledge in LLMs. For evaluation, the authors selected a subset of 2,000 prompts where the target attribute was a single token and correctly predicted by the base models. The methods were tested on two open-weight LLMs: LLaMA 3.1 8B and Qwen3 8B.
Key Insights and Ideas
- Non-Contiguity is the Norm: Factual retrieval paths are sparse. Models routinely skip intermediate layers during factual computation, meaning the process is non-contiguous.
- The Multi-Layer Requirement: Factual retrieval is never a single-layer "lookup." It requires a multi-step sequence of transformations across several layers to construct the final attribute.
- High Computational Redundancy: For the vast majority of facts, models maintain multiple, functionally equivalent paths. If the primary, compact path is disrupted, the model dynamically activates a longer, deeper "backup" path to retrieve the same information.
- Resolving the Editing Paradox: This distributed nature explains why model editing works away from storage sites. Editing succeeds because it intervenes at a necessary transformation stage along the computation path, rather than targeting a single localized factual repository.
Key Results
- Early Completion: In natural runs, factual computation finishes early. The average termination layer ($\ell_{attr}$) is 4.61 for LLaMA 3.1 8B (out of 32 layers) and 7.97 for Qwen3 8B (out of 36 layers).
- Pervasive Alternative Paths: The authors discovered alternative, functionally equivalent paths in 80.1% of cases for LLaMA and 82.7% for Qwen.
- Primary vs. Backup Characteristics: Primary paths are highly compact (mean length of 5.91 layers for LLaMA). Alternative paths are universally longer (mean length of 9.47 for LLaMA) and terminate much deeper in the network (shifting to an average depth of 13.93 for LLaMA and 18.06 for Qwen).
Limitations and Future Work
While this work clearly demonstrates the existence of redundant computation paths, the authors note that their greedy search strategy only establishes a lower bound on model redundancy; an exhaustive combinatorial search might reveal even more alternative paths.
Additionally, because these alternative paths are uncovered via mechanistic interventions (patching), it remains an open question whether an unperturbed LLM naturally utilizes these deeper circuits during standard inference, or if they represent latent, degenerate remnants of the training process. Future work will need to explore how these redundant paths behave during dynamic, real-world generation.
Summary updated