A language model can often answer "Who directed Inception?" and "Where was Christopher Nolan born?" correctly and still fail "In what city was the director of Inception born?" This is the multi-hop reasoning gap. Retrieval systems patch it by writing intermediate reasoning text and searching again, which is slow, or by mapping the model's internal activations straight into a search engine, which is fast but opaque. Neither approach tells us why the composed question fails. Systems neuroscience has a tool built for this kind of question. Demixed principal component analysis (dPCA) splits a population's activity into parts that depend on each experimental factor separately and parts that depend only on how the factors combine. We propose to run language models through a factorial "experiment" of composed questions, and to use dPCA to isolate the interaction subspace: the activity that depends jointly on the first entity and the first relation, and therefore on the bridge entity ("Nolan"), but not on either factor alone or on how the question is worded.
The claim is twofold. Scientifically, we claim that how well the bridge entity can be decoded from this interaction subspace, layer by layer and item by item, separates three failure modes: the bridge was never formed, the bridge was formed but not used, or the bridge was used but the second fact is missing. That diagnostic predicts which composed questions fail. Practically, we claim that a retrieval query read only from the interaction subspace transfers better than a query read from the raw hidden state to unseen question wordings and harder benchmarks, because dPCA explicitly removes wording-driven variance. The second claim is the one most at risk, and the experiments are designed so that it can fail cleanly.
Core Claim and What Is New
dPCA, as introduced in Demixed Principal Component Analysis and extended in Demixed principal component analysis of neural population data, averages population activity over subsets of experimental factors ("marginalization", as in an analysis of variance). It then finds linear readout axes, allowed to be non-orthogonal, that each reconstruct one marginal while ignoring the others. These papers supply the method. They also supply the motivating observation that 70–90% of population variance is condition-independent, meaning it does not depend on stimulus or decision. Plain PCA therefore lets that dominant variance swamp the task signal, while 14 dPCA components still capture 90% of variance against PCA's 91.7%. Their stated limitations constrain our design. dPCA is linear, it needs discrete and balanced factor bins, and subspaces can drift over time. We respect the factorial requirement by constructing balanced question sets rather than fighting it. We treat drift by fitting per layer and per token position rather than assuming one fixed subspace.
On the language side, Optimizing Multi-Hop Document Retrieval Through Intermediate Representations (L-RAG) is the closest work. It shows with logit-lens analysis that middle "processing" layers resolve intermediate hops. It then trains an MLP from raw middle-layer states into the Contriever embedding space, retrieving next-hop documents in about 2.8 seconds per query against more than 20 seconds for iterative methods on 2WikiMultiHopQA. L-RAG never decomposes the hidden state and never asks which part of it carries the bridge. Our proposal differs in its object, which is the entity×relation interaction marginal, and in its goal. The goal is first a diagnostic of composition failure and only second a retriever. The retrieval claim is narrow: invariance and data efficiency under held-out wording, not raw in-distribution accuracy.
Research Program
Factorial question construction. We build balanced factorial sets from Wikidata-style triples using the compositional and inference templates of Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps. There are four factors. The first entity $e$ is drawn from a pool of films or people. The first relation $r_1$ is, for example, director or producer. The second relation $r_2$ is, for example, birthplace or birth year. The surface template $w$ covers several paraphrases. Every cell is filled. The bridge entity $b = r_1(e)$ is a function of the $(e, r_1)$ cell, so its signal must appear in the interaction marginal $X_{e r_1}$ in the decomposition
$$X = \bar{X} + X_w + X_e + X_{r_1} + X_{r_2} + X_{e r_1} + \dots,$$
where $X$ is the matrix of hidden states at a chosen layer and token position across all cells, $\bar{X}$ is the grand mean, and each subscripted term is the variance attributable only to that factor or factor combination. Each item also has its two single-hop questions, so the model's knowledge of each fact is measured separately.
Diagnostic. For each open-weight model and each layer, we fit dPCA with cross-validated regularization, as in the original method. We then measure how accurately a linear decoder on the $X_{e r_1}$ components identifies $b$ on held-out items. Restricting to items where both single hops are answered correctly isolates the composition gap from missing knowledge. We sort these items into three groups: bridge not decodable (never formed), bridge decodable but the composed answer is wrong (formed, not used), and bridge decodable with the composed answer right. A causal check follows. We patch the interaction-subspace projection from an item the model answers correctly into a matched item it fails, and ask whether the answer or the retrieved document follows.
Retrieval readout. We train a linear map from the interaction components to the Contriever embedding of the hop-2 gold passage, using the same training signal as L-RAG. We then plug it into a single-pass retrieval pipeline on standard benchmarks. Because factorial labels exist only in the constructed set, dPCA is fit there once and the resulting fixed projection is applied to natural questions. This transfer is itself a test of whether the subspace is general.
Baselines
The naive reference point is one-shot retrieval on the original question followed by answering, alongside no retrieval at all. This shows the size of the gap being closed.
The strongest low-latency competitor is the L-RAG MLP on raw middle-layer states (Optimizing Multi-Hop Document Retrieval Through Intermediate Representations). It reports 0.352 accuracy on MuSiQue, the highest among the methods it compares. It is the right competitor because it uses the same information source, one forward pass, and the same retriever target. We will reimplement it on the same base model, same layer search, same Contriever index, and same training pairs. It will receive an equal hyperparameter search budget, counted in configurations tried. Beating it means equal or better MuSiQue accuracy in-distribution and a statistically clear advantage on held-out templates and on transfer to MuSiQue when both are trained only on 2WikiMultiHopQA-derived data.
The strongest iterative competitors are IRCoT (Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions), which reports 11–21 points of retrieval recall gain and up to 15 F1, and DRAGIN (DRAGIN: Dynamic Retrieval Augmented Generation based on the Real-time Information Needs of Large Language Models). DRAGIN reports 2WikiMultiHopQA F1 of 0.3931 with LLaMA2-13B against FLARE's 0.3076. We run DRAGIN on LLaMA2-13B to stay comparable to that number. We do not expect a single-pass method to beat these on accuracy. The bar is to close a stated fraction of the gap between one-shot retrieval and DRAGIN at L-RAG-like latency.
The ablation that isolates the key ingredient is a ridge regression on the full raw state, trained on the same factorial data and labels as dPCA. We add two further ablations: dPCA with the interaction marginal removed, keeping only the $e$ and $r_1$ main effects, and dPCA fit on shuffled factor labels. A further control removes the top plain principal components globally before the MLP. This tests whether "drop the dominant condition-independent variance" alone explains any gain.
Why not just…?
Why not just train a supervised linear probe or ridge map on the raw hidden state? This is the real threat. Given the same labels, a supervised linear map can learn to ignore wording variance on its own, and the ~75% decoding result in Demixed principal component analysis of neural population data shows that dPCA decoders match supervised classifiers rather than beat them. We therefore make no in-distribution accuracy claim for dPCA. Its distinct offerings are the explicit separation of interaction from main effects, which a probe for $b$ does not give, and potentially better behavior with few training items and unseen templates. The matched ridge ablation, run across training-set sizes and held-out templates, settles this. If ridge ties everywhere, the retrieval claim is dropped and the contribution is the diagnostic alone.
Why not just use plain PCA or remove the top principal components? PCA is blind to the factors. The original dPCA work shows its components mix stimulus and decision, and the dominant condition-independent variance is exactly what PCA keeps first. Global component removal is cheap, so it is included as a control. If it matches dPCA on retrieval, the "demixing" story reduces to denoising, and we will report that.
Why not just use the L-RAG MLP and probe it afterwards? An MLP fit end to end can succeed through shortcuts, such as matching on surface entities. Inspecting it afterwards cannot tell us whether the bridge was formed. ♫ MuSiQue: Multihop Questions via Single-hop Question Composition shows that such shortcuts are widespread: single-hop models lose 30 F1 there relative to HotpotQA. The factorial design plus the interaction marginal asks a question the MLP cannot answer. Transfer to MuSiQue tests which readout depends less on shortcuts.
Why not just use chain-of-thought retrieval? It works, but costs more than 20 seconds per query on 2WikiMultiHopQA, per L-RAG, and it cannot explain failures inside the model. We keep it as an accuracy ceiling, not as a rival hypothesis.
Proposed Experiments
The first decisive check is predictive. On items where both single hops are correct, interaction-subspace bridge decodability at the best middle layer should predict composed-answer correctness better than raw-state probe confidence does. We measure this with held-out AUC across at least two model sizes. The second check is causal. Patching the interaction projection should flip a measurable share of failed items to the donor's bridge, and patching an equal-dimensional random subspace should not. The third check is retrieval transfer. We train on 2WikiMultiHopQA-derived factorial data with some templates held out, then evaluate hop-2 recall@k and answer F1 on held-out templates, on full 2WikiMultiHopQA, on MuSiQue, and on HotpotQA (HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering), reporting latency per query. All of this runs against the ridge, L-RAG, and DRAGIN baselines above, at matched training data and tuning budget.
The outcome that would most change our minds is the following: decodability of the bridge does not predict failure, or ridge matches dPCA on held-out templates. In the first case the linear-subspace view of composition is wrong for these models. In the second, demixing adds nothing beyond supervision.
References
- Demixed Principal Component Analysis
- Demixed principal component analysis of neural population data
- Optimizing Multi-Hop Document Retrieval Through Intermediate Representations
- Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps
- Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
- DRAGIN: Dynamic Retrieval Augmented Generation based on the Real-time Information Needs of Large Language Models
- ♫ MuSiQue: Multihop Questions via Single-hop Question Composition
- HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering