Selecting Multi-Hop Retriever Training Data by Hop Type, Not by Topic

When a question-answering system pulls its facts from an external index such as Wikipedia, the training data for its retriever no longer has to teach it facts. It only has to teach the model how to move from one piece of evidence to the next: from a film to its director, from a director to a birthplace. We propose to curate that training data for coverage of these moves, which we call hop transitions, rather than for coverage of question topics. Each transition can be measured cheaply as the difference between averaged text embeddings. Averaging token vectors (mean pooling) loses word order, but the result is additive, so subtracting the embedding of the question from the embedding of the document the next hop must reach gives a vector that roughly encodes the relation being crossed. Picking a small, diverse set of examples in this offset space should train a next-hop retriever as well as the full dataset, or better.

The opportunity sits between two results that have not met. Representation-level multi-hop retrieval works: L-RAG trains a small network on middle-layer hidden states of a language model to fetch second-hop documents in a single forward pass. It reports the best accuracy among its compared methods on MuSiQue (0.352) at about 2.8 seconds per query, against more than 20 seconds for iterative query generation on 2WikiMultiHopQA. Small curated training sets also work: on general instruction tuning, SHED and the experimental-design selection of Bhatt et al. match or beat full-data training with a fraction of the examples. The file names the risk of combining them. Selection methods measure diversity in a generic embedding space that may not reflect multi-hop dependencies, so they may favor "irrelevant diversity over necessary reasoning chains." We turn that risk into the hypothesis under test.

Core Claim

For a retriever that queries an external knowledge index, the useful diversity in training data lies in the space of hop transitions, not in the space of question content. Take a training example $i$ with question $q_i$, first-hop evidence $d^{(1)}_i$, and second-hop target $d^{(2)}_i$. Let $\bar e(\cdot)$ be the mean-pooled output of a frozen retrieval encoder. We define the transition vector

$$t_i = \bar e\big(d^{(2)}_i\big) - \bar e\big(q_i\big),$$

and select a subset $S$ of size $k$ by maximizing the facility-location objective, which rewards a subset that has a close representative for every example in the pool:

$$F(S) = \sum_{i \in \text{pool}} \max_{j \in S} \kappa(t_i, t_j),$$

where $\kappa$ is cosine similarity. This is the submodular selection used by Bhatt et al., unchanged except for the space it runs in. Our prediction is that at budgets of 5 to 10 percent of the training pairs, a retriever trained on transition-selected data matches full-data second-hop recall in-domain and exceeds it when moved to a different multi-hop dataset. The same selection run on question embeddings should do clearly worse.

The reasoning behind this comes from the external-knowledge direction. The usual objection to aggressive curation is that it throws away long-tail facts. For a retriever over an external index that objection mostly disappears, because the facts stay in the index. Closed-book models fail on long-tail knowledge: PaLI-X reaches 10.8% on INFOSEEK without a knowledge base (Hu et al.). What the training set must supply instead is coverage of relation types and of how hops compose. If this view is correct, data quality for retrieval-dependent systems means covering transitions, and topic coverage is largely wasted budget.

Why averaged embeddings, and where they might fail

The geometric bet borrows from work on averaged embeddings. Mean pooling acts as a usable semantic summary and often as a denoiser. CRISP collapses token clusters into mean centroids inside a multi-vector retriever and slightly improves BEIR NDCG@10 (54.5 vs. 54.3) while cutting the number of vectors by 2.9x. Feldman and Artzi find that plain mean pooling beats learned compression tokens for context compression. Both papers support using averages as cheap, faithful summaries, and we reuse that view as our measurement tool.

Two findings constrain the design. Dufter et al. show that averaged static vectors can serve as a knowledge base, reaching 41.2% on LAMA against BERT's 39.6%. They also show that averaging ignores the relation in a query unless the relation is imposed externally, and that it degrades as entity names grow longer. Feldman and Artzi explicitly leave open whether mean pooling keeps the fine-grained bridge entities that multi-hop chains need. Both warn that a single averaged vector may dilute the relation. The offset construction is our attempt to separate the relation from the content: shared topical content cancels in the subtraction, and what remains is closer to the direction of the move. Whether the remainder actually encodes relations is an empirical question, and the first experiment tests it directly.

Research Program

We build three pieces. The first is the transition embedding, computed with a frozen Contriever-style encoder; this is the retrieval space L-RAG maps into. We study two variants: the question-to-target offset defined above, and an evidence-to-target offset that subtracts $\bar e(d^{(1)}_i)$ instead of the question embedding. The second piece is the selector. It runs greedy facility-location maximization over the pool of 2WikiMultiHopQA, HotpotQA, and MuSiQue training pairs at budgets of 1, 5, 10, and 25 percent. The third piece is the downstream learner, an L-RAG-style mapping network from middle-layer states to the retrieval space. We train it on each subset with the training recipe held fixed.

We then add a reader that is identical for every condition, so answer accuracy changes only through retrieval. Because the reader and the index never change, any difference between conditions comes from which training pairs the mapping network saw.

A secondary, clearly labeled stretch study carries the idea to multimodal bridging. On MMhops (MMhops-R1), the bridge runs from an entity recognized in an image to a text fact. There we compute offsets between the mean-pooled embedding of the image entity's description and the target passage. We then select training trajectories for a supervised warm-up before any reinforcement learning. This study tests generality only; the core claim stands or falls on the text experiments.

Baselines

The naive reference points come first. One is training the mapping network on the full training set, which is the L-RAG setting; its reported anchor is 0.352 accuracy on MuSiQue (L-RAG). The other is a uniformly random subset at each budget, averaged over five seeds. Beating the full set at 10 percent of the data is the strong win. Beating random at every budget is the minimum requirement.

The strongest existing selection method is facility location over question or question-plus-answer embeddings, as in Bhatt et al.. Its 45k selected pairs beat 90k random pairs, with about a 2% gain on MMLU and BBH and a 52.8% AlpacaEval win rate. We found no reported result on Lacuna for facility-location or Shapley-based selection applied to multi-hop retriever training. This baseline is the direct test of our claim, because it shares our objective, kernel, encoder, and budget and differs only in the space where diversity is measured. We run it with the same greedy solver, the same frozen encoder, identical mapping-network hyperparameters, and the same number of training steps. Counting as a win requires second-hop Recall@5 at least two points higher on the cross-dataset transfer split, consistent over five seeds.

We also include SHED in clustered-Shapley form on question embeddings. SHED reports that 4,000 curated WizardLM samples reach 35.92% against 33.16% for the full 70k. It gets the same selection compute budget, measured in encoder forward passes plus proxy evaluations, so that a more expensive selector gains no hidden advantage.

The key ablation swaps the space while keeping everything else fixed. The selection runs on target-document embeddings alone, on question embeddings alone, on their concatenation, and on the offset. If concatenation matches the offset, the subtraction is not doing the work.

Why not just…?

Why not just train on all the data? For instruction tuning, SHED and Bhatt et al. show that full data is often not optimal. Neither result covers retrieval, though, and the full-data baseline is in every table. If full data wins at every budget, the claim fails. The core claim predicts a specific pattern instead: full data should be matched in-domain and beaten out of domain, where a training set dominated by a few frequent relation types should overfit the most.

Why not just balance by relation or composition type? This is the most serious threat. 2WikiMultiHopQA has labeled question types, and ReasonVQA shows that balancing answer distributions removes shortcut memorization. Wherever such labels exist, stratified sampling by type is nearly free and may equal our method. We therefore run label-stratified sampling as an explicit baseline on 2WikiMultiHopQA and accept the outcome. If it ties, the contribution narrows to a label-free way of recovering the same structure, and its value then rests on MuSiQue and HotpotQA, where no clean type labels exist and our method must beat random and question-space selection. We also measure directly whether clusters in offset space recover the known 2Wiki types.

Why not just select the hardest examples? Following DART-Math, which favors difficult verified examples over volume, one could keep the pairs where an off-the-shelf retriever misses the second hop. We include this baseline at matched budgets. We expect hard-only selection to pile into a few difficult relation types and to transfer worse. A combined selector, facility location restricted to the hard half, tests whether hardness and coverage add up.

Why not just skip retriever training and use iterative chain-of-thought retrieval? Methods such as IRCoT need no trained mapping, but they pay a decoding loop per hop: more than 20 seconds against about 2.8 seconds in L-RAG's 2WikiMultiHopQA measurement. We report one iterative baseline for accuracy context. The proposal targets the cheap, single-pass setting, where training data is the main remaining lever.

Proposed Experiments

The first decisive check is intrinsic and cheap, and it runs before any training. On 2WikiMultiHopQA, we cluster training examples in offset space and in question space and measure agreement with the gold question-type labels using normalized mutual information. If offset clusters do not track types better than question clusters, the averaging-cancels-content premise is wrong. In that case we would move to entity-span-masked averaging before going further, guided by Dufter et al.'s finding that averaging handles relations poorly.

The main experiment trains the mapping network on each selected subset at the four budgets with five seeds. We report second-hop Recall@5 and Recall@20 over the Wikipedia index and end-to-end answer exact match and F1 with the fixed reader. HotpotQA results use the standard distractor and full-wiki settings (HotpotQA). The transfer test trains on one dataset's selected subset and evaluates on the other two. This is where the claim should show most clearly. We also check whether a subset chosen with one encoder still works when the mapping network sits on a different language-model backbone, mirroring SHED's transfer finding.

The mean-pooling robustness check repeats selection with the offsets computed from first-token embeddings and from CRISP-style cluster centroids instead of plain means. This shows whether the dilution that Feldman and Artzi flag for bridge entities hurts the selection signal.

The multimodal stretch study selects MMhops bridging trajectories at 10 percent for warm-up and compares against random and full-data warm-up under an identical later training budget. MMhops-R1's reported 51.35% bridging accuracy is the reference, not a target we expect to exceed at reduced data.

The result that would most clearly confirm the proposal is offset-space selection at 10 percent beating both full data and question-space facility location on cross-dataset second-hop recall. The result that would most clearly refute it is a tie with random selection.