Multi-hop questions such as "In what city was the director of Inception born?" break retrieval systems in a specific place. The passage that answers the question is about Christopher Nolan, and the question never names him. Graph-based retrieval pipelines, which link passages through shared entities, should handle this, because Nolan's node is one edge away from Inception's. In practice they pick their starting points and their node relevance scores by comparing a frozen sentence embedding of the question to a frozen embedding of each node's own text. The bridge node's own text shares little vocabulary or meaning with the question, so it scores low. The traversal then either never reaches it or reaches it only after spending its budget on noise.
We propose a small change at that step. Each node keeps its pre-trained text embedding unchanged, and we add a learned, low-rank correction built from its graph neighbors' embeddings. The correction lets a node "borrow" the vocabulary of the entities it connects to, so that Nolan's vector moves toward questions about Inception. A regularizer keeps the corrected vector close to its frozen anchor so that scarce multi-hop training data cannot distort the embedding geometry. The question encoder and text encoder are never updated. The correction is inductive, so a changed corpus only requires recomputing neighbor sums, not retraining. The central question is empirical: does learned, anchored, structure-conditioned node scoring beat both parameter-free neighbor averaging and full encoder fine-tuning, especially when questions are phrased in vocabulary that differs from the corpus's?
Core Claim
The multi-hop gap and the vocabulary gap meet at the seed-and-prize scoring step of graph retrieval. Neither the text encoder nor the graph search alone can close them there. The text encoder cannot see that a node is relevant through its neighbors. Graph search cannot recover a node it never scored highly. The fix belongs in the node's retrieval vector. How that fix is trained matters, because the obvious alternative of fine-tuning the encoder on a few thousand multi-hop examples is exactly the regime where updated embeddings pull away from unseen ones and generalization collapses. We claim that an anchored residual keeps most of the benefit of learning while holding the transfer robustness of frozen features. The benefit shows up as higher recall of bridge nodes at the seed stage, and the robustness shows up under domain shift and lexically rewritten questions.
Concretely, let $e_v$ be the frozen text embedding of node $v$, let $\mathcal{N}(v)$ be its graph neighbors, and let $q$ be the frozen question embedding. The node's retrieval vector is
$$ z_v = e_v + A B^\top \sum_{u \in \mathcal{N}(v)} \alpha_{vu}\, e_u , $$
Here $A, B$ are thin $d \times r$ matrices with small rank $r$, and $\alpha_{vu}$ are attention weights computed from $e_v$, $e_u$ and the text of the connecting relation. The weights do not depend on the question, so $z_v$ can be precomputed and indexed. Training minimizes a contrastive loss that ranks gold bridge and answer nodes above sampled nodes under $q^\top z_v$, plus an anchor term
$$ \mathcal{L}_{\text{anchor}} = \lambda \sum_v \big\| \mathrm{sm}(z_v^\top E_{\text{ref}}) \,\big\|\, \mathrm{sm}(e_v^\top E_{\text{ref}}) \big\|_{\mathrm{KL}} , $$
$E_{\text{ref}}$ is a fixed sample of frozen reference embeddings, and $\mathrm{sm}$ is a softmax. The term asks that each corrected node keep roughly the same neighbors in the original space that it had before correction. It is a geometric, not merely norm-based, anchor.
Where This Sits in Prior Work
Three papers motivate the proposal by exhibiting the failure. G-Retriever scores nodes by nearest-neighbor similarity to frozen SentenceBERT embeddings and then solves a Prize-Collecting Steiner Tree, which picks a small connected subgraph that collects high-scoring nodes. A bridge entity with no similarity to the question receives no prize, so the tree has no reason to include it. Query-Aware Flow Diffusion for Graph-Based RAG (QAFD-RAG) reweights edges by query alignment and diffuses relevance from seed nodes, with recovery guarantees. Its seeds still come from keyword extraction and frozen similarity, so its guarantees assume the seeds are right. Beyond Explicit Edges (INSES) adds "virtual edges" between semantically similar nodes when the graph is sparse. It uses frozen similarity and does not ask how adapting that similarity would behave. Our correction is a drop-in replacement for the node scorer in all three, and we do not modify their search procedures.
Three lines supply the method or constrain it. Learning Word Representations from Scarce and Noisy Data with Embedding Subspaces shows that updating individual embedding vectors from scarce data tears apart updated and non-updated words, and that restricting updates to a subspace avoids this. Our low-rank residual applies that idea at the node level. Zero-Resource Cross-Domain NER finds similar overfitting to source vocabulary when embeddings are unfrozen. Both results predict that unanchored fine-tuning will fail under transfer. STAG fuses GNN neighborhood signals with text features under a KL term that keeps the fused distribution near the text-only one, over a frozen codebook. We reuse that anchoring idea, but STAG targets single-node classification. We target the retrieval score that decides which nodes a multi-hop search visits. GLEM shows that structure can be distilled into text representations without joint backpropagation. We go further and keep the encoder entirely frozen, which GLEM does not.
The strongest end-to-end competitors take the opposite bet from ours. GRITHopper fine-tunes a 7B decoder to retrieve hop by hop from a flat index. MoPo distills query-focused summaries into a fully trained dense retriever. L-RAG maps middle-layer LLM states into a frozen retriever's space. All three adapt the query side and ignore graph connectivity. We adapt the node side through structure and keep all text encoders fixed. That difference is what makes corpus updates cheap and, we predict, transfer safer. Hierarchical Lexical Graph attacks the lexical gap by rewriting the index into propositions and LLM-derived topics, with many LLM calls per document. Our correction costs one sparse matrix product per node.
Research Program
The first component is shared infrastructure. For each benchmark corpus we build an entity–passage graph by open information extraction and entity linking, following the construction style of the graph-RAG baselines, and embed all nodes and questions with one frozen encoder (Contriever or a BGE-class model). We fix the graph so that every method sees identical structure.
The second component is the anchored residual itself, trained on the HotpotQA and 2WikiMultiHopQA training splits. Positives are nodes attached to gold supporting passages, and bridge nodes are marked explicitly where the annotation allows. We sweep the rank $r$ and the anchor weight $\lambda$ on a held-out development split drawn from the training datasets only, never from the transfer targets.
The third component is a lexically shifted evaluation set. We rewrite MuSiQue-Ans development questions with an LLM, replacing content words and entity surface forms with paraphrases or descriptions while preserving the answer. We filter by measured content-word overlap with the gold passages and have annotators verify answer preservation on a sample. We report the overlap reduction achieved. The observation by Yang et al. that many correct open-domain answers share no content words with the question is the motivation, and this set lets us measure that regime directly on multi-hop data. We choose MuSiQue because its construction blocks the single-hop shortcuts that make HotpotQA an unreliable test of bridge retrieval.
The fourth component is analysis. We measure how far each method moves node vectors, using mean change in the original-space neighbor ranking. We relate that drift to the transfer loss, which directly tests the direction-1 claim that drift is what breaks generalization.
Baselines
The naive reference is the unmodified frozen encoder, with $z_v = e_v$, plugged into QAFD-RAG seeding and G-Retriever prizes. It is the configuration those papers ship, so every gain is measured against it.
The strongest graph-RAG system is QAFD-RAG. The file reports it at 73.42 multi-hop QA F1 on HotpotQA versus 58.67 for HippoRAG and 33.40 for GraphRAG (QAFD-RAG). No separate MuSiQue number was found on Lacuna for QAFD-RAG. We rerun it on our graphs with the same reader LLM, changing only the seed scorer. Beating it means a statistically significant F1 gain on MuSiQue and on the rewritten set, while staying within noise on HotpotQA. INSES is a second graph baseline, reported at 0.68 EM on HotpotQA against 0.61 for SiReRAG (INSES). We substitute our vectors for its virtual-edge similarity.
The strongest learned retrievers are MoPo, reported at 82.20 EM@100 on HotpotQA against 79.70 for MDR (MoPo), and L-RAG, reported at 0.352 accuracy on MuSiQue (L-RAG). They are run with their released training recipes on the same training data. We compare at matched retrieval budget, and we also report wall-clock latency and the cost of re-indexing after a corpus change. GRITHopper is included where compute permits. We note its 7B fine-tuning cost as a real asymmetry rather than hiding it.
Three ablations isolate the proposal's ingredients. The first is parameter-free propagation, $z_v = e_v + \beta \cdot \mathrm{mean}_{u \in \mathcal{N}(v)} e_u$, with $\beta$ tuned on the same development split with the same number of trials as our hyperparameters. It tests whether learning is needed at all. The second is the same residual with $\lambda = 0$, which tests whether the anchor matters. The third fine-tunes the text encoder end to end on the same supervision, with a matched tuning budget, which tests the freezing decision. Seed-stage bridge recall is our primary diagnostic metric. No reported result on this metric was found on Lacuna for any baseline, so all seed-stage numbers will be produced by our reruns.
Why not just…?
Why not just average each node with its neighbors, training-free? This is the most serious threat, because it also lets a bridge node borrow neighbor vocabulary. Our reason to expect it to fall short is dilution. Entities like countries or years have hundreds of neighbors, and uniform averaging blurs them toward a generic centroid. This is the over-smoothing that semantically grounded message passing is meant to prevent (Wang et al.). The learned attention can instead pick the one relation that matters. That expectation is not evidence. If tuned averaging comes within noise of the learned residual on MuSiQue and the rewritten set, we will report that the contribution is the placement of the fix, not the learning. The paper would then be about parameter-free structural seeding.
Why not just rewrite the query, as RaFe does? RaFe trains small rewriters from reranker feedback and reports 2–3% downstream QA gains. A rewriter can only add terms it can infer from the question. It cannot know the director's name without either parametric knowledge or a retrieval step. The bridge problem is precisely the missing name. We run a RaFe-style rewriter in front of the frozen seeder, and in front of our seeder, to check whether the two gains add up.
Why not just iterate retrieval with chain-of-thought? IRCoT recovers bridge entities by generating them, gaining up to 15 F1 points. It pays several sequential LLM calls per question; L-RAG reports iterative generation exceeding 20 seconds per query against about 2.8 seconds for its own method. It also inherits cascading errors when an early hop is wrong. Our method is a single-pass scoring change, so the fair comparison is accuracy at fixed latency. We also combine the two, since better seeds should shorten IRCoT's chains.
Why not just fine-tune the retriever end to end? This is what GRITHopper and MoPo do, and on in-distribution HotpotQA they may win. The direction-1 evidence (Astudillo et al., Liu et al.) predicts they lose ground under transfer and vocabulary shift. The fine-tuning ablation and the drift analysis test this directly. If fine-tuning holds up on the transfer targets, our claim fails and we will say so.
Proposed Experiments
The first decisive check is seed-stage bridge recall@k on MuSiQue-Ans, on the lexically rewritten MuSiQue set, and on 2WikiMultiHopQA. It is computed before any graph search, for frozen, averaged, unanchored, anchored and fine-tuned scorers. The prediction is that anchored and fine-tuned tie in distribution, and that anchored beats fine-tuned and averaged on the rewritten set and on 3–4 hop MuSiQue questions.
The second check is end-to-end QA F1 and EM when these scorers are plugged into QAFD-RAG, G-Retriever and INSES with a fixed reader. It asks whether seed gains survive the search.
The third check is transfer. Models trained only on HotpotQA and 2Wiki are evaluated on MuSiQue, on MultiHop-RAG, and on the UltraDomain vertical domains that QAFD-RAG uses. For UltraDomain we score with its own comprehensiveness and logicality protocol. Here the anchor should matter most.
The fourth check is the drift–transfer relation. Across all trained variants and $\lambda$ values, we plot neighbor-rank drift against the transfer-set F1 drop. A monotone relation would support the mechanism we propose. A flat one would mean the anchor helps for some other reason, or not at all.
References
- G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answering
- Query-Aware Flow Diffusion for Graph-Based RAG with Retrieval Guarantees
- Beyond Explicit Edges: Robust Reasoning over Noisy and Sparse Knowledge Graphs
- Learning Word Representations from Scarce and Noisy Data with Embedding Subspaces
- Zero-Resource Cross-Domain Named Entity Recognition
- Quantizing Text-attributed Graphs for Semantic-Structural Integration
- Learning on Large-scale Text-attributed Graphs via Variational Inference
- GRITHopper: Decomposition-Free Multi-Hop Dense Retrieval
- Momentum Posterior Regularization for Multi-hop Dense Retrieval
- Optimizing Multi-Hop Document Retrieval Through Intermediate Representations
- Hierarchical Lexical Graph for Enhanced Multi-Hop Retrieval
- WikiQA: A Challenge Dataset for Open-Domain Question Answering
- ♫ MuSiQue: Multihop Questions via Single-hop Question Composition
- Towards Graph Foundation Models: Training on Knowledge Graphs Enables Transferability to General Graphs
- RaFe: Ranking Feedback Improves Query Rewriting for RAG
- Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions