Questions like "In what city was the director of Inception born?" break ordinary retrieval-augmented generation, where a language model answers using documents fetched by a search step. The first search can find who directed the film, but it cannot find the birthplace, because the name "Christopher Nolan" is not in the question. Today's fixes either have the model write out each reasoning step as text and search again, which is slow and lets one wrong generated name derail everything downstream, or they train a projection from the model's internal state into the search engine's embedding space, which is fast but opaque. We propose a third route. We look for a small set of attention heads (the individual components inside a Transformer layer that decide which earlier tokens each position looks at) whose job in multi-hop questions is to point from the unresolved part of the question to the intermediate "bridge" entity inside the evidence already retrieved. If such heads exist, the next search query can be copied verbatim from the span they point at, using one forward pass and no text generation. How sharply they point also tells us whether the previous hop actually succeeded.
The opportunity matters because interpretability work has shown that retrieval-like behavior in language models is concentrated in very few heads, but every existing head-detection method scores heads on matching the given query to a passage. That is exactly the capability that fails in multi-hop questions. Nobody has asked whether some heads specialize in the hand-off from one hop to the next, or whether those heads differ from the known query-matching heads.
Core Claim
Our claim is that in instruction-tuned 7–13B models, the binding of a question's open slot to a bridge entity in retrieved text is carried by a sparse set of heads, which we call bridge heads. These heads are largely distinct from query-focused retrieval heads. Their attention gives a grounded pointer that supports iterative multi-hop retrieval at close to single-pass cost, with fewer cascading errors than generated intermediate queries. The claim can fail in two clean ways. If bridge heads overlap heavily with existing retrieval heads, the "hop-specific" part is false and existing heads suffice. If a two-token greedy decode of the bridge entity works just as well, the pointer adds little beyond latency.
Relation to Prior Work
Three works motivate the proposal by showing that internal attention carries a retrieval signal, while leaving the hop transition unstudied. DRAGIN uses attention to decide when to retrieve and builds the query from the tokens with the highest attention, but it averages over all heads. The authors report that LLaMA2-7B is easily overwhelmed by long retrieved context. We replace that average with heads selected for one specific function. QRHEAD scores heads by how much attention flows from query tokens to relevant context, and shows these heads differ from copy-paste "needle" heads. That finding is the template for our claim that bridging is in turn distinct from query matching. However, QRHEAD is single-step by design, and its authors leave circuit-level causal analysis open. CoRe-R selects heads that separate relevant passages from hard negatives and finds that about 8 heads, concentrated in middle layers, beat all heads. It even reports on MuSiQue, but only for re-ranking a fixed candidate set. We reuse its contrastive scoring idea, applied to entities within passages rather than whole passages.
L-RAG is the closest competitor on the systems side. It shows that middle-layer hidden states already encode the next-hop intent and trains a projection into Contriever space, retrieving the hop-2 document at about 2.8 seconds per query versus more than 20 seconds for an iterative generate-then-retrieve loop. It treats the layer as a single vector and is built around two hops. We differ in reading a token span out of specific heads, which makes the query auditable and extensible to k hops, and in requiring no trained projection. On the interpretability side, Sun et al. trace closed-book multi-hop circuits and attribute failures to heads that misroute the intermediate entity. That supports our hypothesis that routing lives in a few heads, but they study only knowledge stored in the weights, not retrieved evidence. Neo et al. show heads acting as context detectors that trigger MLP neurons, and Voita et al. show most heads can be pruned while specialized ones carry the load. Both make sparsity a reasonable prior rather than a guarantee. Contrastive Covariances supplies an optional method for explaining what query-key feature drives a bridge head.
Research Program
The first stage is detection. MuSiQue (paper) is built by composing single-hop questions and annotates each sub-question, its supporting paragraph, and the intermediate answer. This gives gold bridge-entity spans inside the hop-1 paragraph. 2WikiMultihopQA (paper) adds structured reasoning triples for a second detection source. We place the question and the gold hop-1 paragraph, plus distractor paragraphs, in context and run one forward pass with no generation. For each head $h$ we compute a contrastive bridge score:
$$ B_h = \mathbb{E}\left[\, A_h(q \to b) - \max_{e \in \mathcal{E}^-} A_h(q \to e) \,\right] $$
Here $A_h(q \to s)$ is the attention mass that head $h$ sends from the question's final token to span $s$. The span $b$ is the gold bridge entity. The set $\mathcal{E}^-$ contains the other entity spans in the same paragraph and in distractors. The expectation runs over training questions. Subtracting the hardest competing entity, rather than averaging over all of them, is borrowed from CoRe-R's hard-negative scoring. It forces a head to prefer the right entity, not just entity-like tokens. We keep the top-k heads, with k swept from 4 to 32, and measure their overlap with QRHEAD heads computed on the same models using the released QRHead code.
The second stage is causal validation. We mean-ablate bridge heads and, separately, QRHEAD heads. We then measure accuracy on two kinds of input: the composed multi-hop questions, and the single-hop sub-questions given the same paragraphs. Bridge heads should mainly hurt the composed questions, and QRHEAD heads should mainly hurt the single-hop ones. In a counterfactual test, we edit the hop-1 paragraph so the bridge entity changes (Nolan to Villeneuve) and check whether the bridge-head pointer and the model's final answer both follow the edit. As an optional add-on, we run a contrastive-covariance decomposition on the top heads, contrasting the correct-relation entity against a same-type wrong-relation entity. This tests whether a low-rank relation-binding direction exists, as opposed to a mere entity-type direction.
The third stage is the retrieval system, which we call bridge-pointer retrieval. At each hop we prefill the question plus the evidence retrieved so far. We take the span of maximal summed bridge-head attention within the newest passage and snap it to token boundaries with a fixed rule: expand to adjacent tokens above half the peak value. We append the span to the question's residual text and send the result to the same retriever used by all baselines, either Contriever or BM25.
The concentration of bridge-head attention, defined as the share of bridge-head attention mass on the top span, serves as a gate. High concentration means proceed to the next hop or answer. Low concentration means the previous retrieval missed and should be retried with the next-ranked passage. This gives the system a principled way to handle 2- to 4-hop MuSiQue chains and to stop, which L-RAG leaves unaddressed. It also directly targets the compounding failure, highlighted by Wang et al., in which one bad intermediate step corrupts everything after it.
Baselines
The naive reference is one-shot dense retrieval with the original question, followed by answer generation over the top passages. It shows the size of the gap every method must close.
The strongest accuracy competitor is IRCoT, which interleaves chain-of-thought sentences with retrieval. Its authors report gains of 11–21 retrieval-recall points and up to 15 F1 over one-step retrieval across MuSiQue, HotpotQA and 2WikiMultihopQA. The file gives those gains but no absolute MuSiQue F1, so we will re-run IRCoT ourselves with the same backbone, retriever, corpus, passage budget per hop, and maximum hop count.
The strongest efficiency competitor is L-RAG. It reports MuSiQue accuracy of 0.352, the highest among the RAG methods it evaluated, and about 2.8 s per query on 2WikiMultiHopQA versus more than 20 s for IterRetGen. We run its released code with its projection trained on the same training split we use for head detection, so the supervision budget matches.
For attention-based dynamic retrieval we run DRAGIN's attention-based query formulation. DRAGIN reports F1 0.3931 on 2WikiMultihopQA with LLaMA2-13B, against 0.3076 for FLARE. We include LLaMA2-13B specifically so this number is reproducible in our setup before we move to newer backbones.
CoRe-R has no reported MuSiQue number on Lacuna, so we compute CoRe heads ourselves and use them as a head-selection alternative.
Three ablations isolate the key ingredient, which is selecting heads for the hop transition rather than for query matching. The first replaces bridge heads with an equal number of QRHEAD heads inside an otherwise identical pointer pipeline. The second uses an equal number of random middle-layer heads. The third averages over all heads, which is essentially DRAGIN's signal in our pipeline.
A final baseline is short greedy decoding, in which the model is prompted to emit only the bridge entity, capped at 8 tokens, which is then used as the query. All methods share the backbone, retriever, corpus, top-k passages per hop, and a fixed hyperparameter search of equal size on the MuSiQue development split.
Beating the field means three things. Bridge-pointer retrieval must match or exceed IRCoT's answer F1 and supporting-paragraph recall on MuSiQue. It must stay within roughly L-RAG's per-query latency. And it must beat the QRHEAD-swap ablation by a margin outside seed and bootstrap variance. If it matches IRCoT but not the ablation, the system result stands but the scientific claim about specialized heads does not.
Why not just…?
Why not just plug QRHEAD or CoRe heads into DRAGIN? This is the obvious move and the most serious threat to the proposal. QRHEAD shows that heads detected on one function can transfer surprisingly far. If query-focused heads already point at bridge entities once hop-1 evidence is present, bridge heads are redundant. We do not argue this away. The overlap measurement and the QRHEAD-swap ablation test it directly, and low overlap together with a clear ablation gap is the result the core claim depends on. If overlap is high, we report that existing retrieval heads already do bridging, which is itself a useful finding, and drop the specialization claim.
Why not just use L-RAG's learned projection? It is fast and already beats the iterative methods it was compared against on MuSiQue. But it outputs an embedding no one can read, it needs training per backbone and retriever, and it was designed for two hops. A pointer to a text span can be checked against the passage, can be fed to BM25 as well as dense retrievers, and supplies a concentration signal for retrying and stopping. Whether these properties buy accuracy, and not just interpretability, is decided by the 3–4-hop MuSiQue split and the retry experiment. That is where we expect a single trained projection to degrade.
Why not just decode the bridge entity in a few tokens? Short decoding costs little, and IRCoT's success suggests that generated intermediate text is useful. The difference is that a decoded name can be hallucinated, while a pointer is restricted to text that actually appears in the retrieved evidence. A pointer also gives a confidence value without an extra calibration step. The short-decode baseline is run under identical conditions. We also report the fraction of hop-2 queries containing entities absent from the retrieved passages, alongside downstream F1. If short decoding ties on both, the honest conclusion is that the head method's advantage is latency only.
Why not just use a bigger model or put everything in a long context? MultiHop-RAG reports that even with ground-truth evidence in context, Llama2-70B reaches only 0.32 accuracy and Mixtral-8x7B 0.36, against 0.89 for GPT-4. QRHEAD reports more than 10% gains from head-based selection over full-context generation on LongMemEval and CLIPPER. Scale and context length alone therefore do not remove the problem in open models. We still include a Llama-3.1-70B full-context reference so readers can see the gap.
Proposed Experiments
Bridge heads are detected on the MuSiQue and 2WikiMultihopQA training splits. End-to-end evaluation covers MuSiQue-Ans, split by hop count, plus HotpotQA (paper) in the full-wiki setting and 2WikiMultihopQA. MultiHop-RAG serves as an out-of-distribution transfer test with no re-detection, and its null queries check whether low pointer concentration correctly signals that no answer exists. Backbones are Llama-3.1-8B-Instruct and Qwen2.5-7B-Instruct, which the QRHead code supports, plus LLaMA2-13B for the DRAGIN comparison. Metrics are answer exact match and F1, supporting-paragraph recall per hop, wall-clock latency and forward passes per query, and the rate at which hop-2 queries contain unsupported entities.
The decisive checks come in a fixed order, because each one depends on the previous. First, ablating the top-16 bridge heads must reduce composed-question accuracy substantially more than single-hop sub-question accuracy, with QRHEAD ablation showing the reverse pattern. Without this double dissociation, there is no distinct mechanism. Second, in the counterfactual edits, the pointer and the final answer must follow the substituted bridge entity in most cases. Third, the pointer system must beat the QRHEAD-swap and random-head ablations on hop-2 recall. Fourth, the gap over L-RAG and short decoding should widen on 3- and 4-hop MuSiQue questions, where retrying and stopping matter. If the gap does not widen there, we state that pointer-based retrieval is a faster, more transparent equivalent of existing methods rather than a more accurate one.
References
- DRAGIN: Dynamic Retrieval Augmented Generation based on the Real-time Information Needs of Large Language Models
- Query-Focused Retrieval Heads Improve Long-Context Reasoning and Re-ranking
- Contrastive Retrieval Heads Improve Attention-Based Re-Ranking
- Optimizing Multi-Hop Document Retrieval Through Intermediate Representations
- Bridging the Language Gap: Uncovering and Aligning Shared Circuits for Multi-Hop Reasoning in Multilingual LLMs
- Interpreting Context Look-ups in Transformers: Investigating Attention-MLP Interactions
- Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned
- Decomposing Query-Key Feature Interactions Using Contrastive Covariances
- ♫ MuSiQue: Multihop Questions via Single-hop Question Composition
- Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps
- Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
- MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries
- HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering