Grounded Bridge Extraction: Using Two-Way Attention to Write the Next Hop's Query

Many questions cannot be answered from one document. "In what city was the director of Inception born?" needs one page to learn that the director is Christopher Nolan and a second page to learn his birthplace. The second page shares almost no words with the question. Today's fix is to have a large language model write out its reasoning ("Inception was directed by Christopher Nolan…") and use that text as the next search query. This works, but it is slow. It also fails badly when the model writes a wrong name, because every later search then goes looking for the wrong thing.

We propose a cheaper and more grounded way to write the next query: copy it out of the document already retrieved. The tool is bidirectional co-attention, a layer that compares every word of the question with every word of a candidate passage in both directions. In the question-to-passage direction, we find passage words the question is "asking about." In the passage-to-question direction, we filter to the subset the question does not already contain. Those words, typically the bridge entity, become the next hop's query verbatim. Because they come from the corpus itself, they use the corpus's own vocabulary, which addresses the word-mismatch problem at the second hop. They also cannot contain invented entities.

Core Claim

In multi-hop retrieval, the hard vocabulary gap lies between the question and the second-hop document, not between the question and the first. That gap is closed most reliably by spans that already appear in the first-hop document. A co-attention scorer exposes exactly the alignment needed to find those spans.

Let $A_{ij}$ be the attention weight from question token $i$ to passage token $j$, normalized over $j$. Let $B_{ji}$ be the reverse weight from passage token $j$ to question token $i$, normalized over $i$, and let $s(j,i)$ be the raw token similarity. We score each passage token $j$ as a bridge candidate with

$$b_j = \Big(\sum_i A_{ij}\Big)\cdot\Big(1 - \max_i \sigma\big(s(j,i)\big)\Big),$$

where $\sigma$ is a sigmoid.

The first factor is high when the question's tokens concentrate attention on $j$. The second factor is high when $j$ has no close match among the question's tokens, meaning it is new information. Contiguous high-$b_j$ spans form the next query, issued to an ordinary index (BM25 or a dense index), so documents stay pre-indexed. The second factor is a novelty term and is computed from the reverse alignment $s(j,i)$. A one-directional scorer has no natural analogue of it, which is why we treat bidirectionality as the key ingredient and test it directly.

Relation to Prior Work

Three lines of work motivate the proposal by documenting the gap.

The generative bridging methods are our main foil. IRCoT (Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions) uses each chain-of-thought sentence as the next query. ITER-RETGEN (Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy) uses a whole draft answer. Both treat the retriever as a black box and bridge vocabulary by generating text. We keep the iterative loop but replace generation with extraction from retrieved text.

RaFe (RaFe: Ranking Feedback Improves Query Rewriting for RAG) trains a rewriter from reranker feedback. It acts before retrieval, so it cannot know a bridge entity it has not seen. We borrow its idea of training from retrieval feedback rather than human labels.

Two closer competitors also extract from retrieved text:

  • Baleen (Baleen: Robust Multi-Hop Reasoning at Scale via Condensed Retrieval) condenses retrieved passages into key sentences and appends them to the query. Its scorer is one-way late interaction, and its condenser is trained on sentence-level supporting-fact labels. We differ in three ways: token-span rather than sentence granularity, an explicit novelty filter from the reverse attention direction, and training from second-hop retrieval success alone.
  • L-RAG (Optimizing Multi-Hop Document Retrieval Through Intermediate Representations) skips text generation by mapping a model's middle-layer states into Contriever's embedding space. Its next-hop query is a dense vector that still inherits the bi-encoder's vocabulary limits. Ours is literal corpus text.

Two papers supply the method. DCMN+ (DCMN+: Dual Co-Matching Network for Multi-Choice Reading Comprehension) provides the pairwise bidirectional matching layer and an ablation in which bidirectional matching beats unidirectional matching. TIASL (Implicit Discourse Relation Recognition using Neural Tensor Network with Interactive Attention and Sparse Learning) warns that dense interaction matrices are noisy without sparsity. We follow it by adding an $l_1$ penalty on $b$, so the selector keeps a few short spans.

Two pieces of prior work constrain the design. Transformer-XH (Transformer-XH: Multi-Evidence Reasoning with eXtra Hop Attention) reasons over an already-built evidence graph and cannot find documents missing from it. The HotpotQA BiDAF baseline (HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering) fell to 17.73 Joint F1 in the open full-Wikipedia setting. Together they show that co-attention used only as a reader over a fixed context does not solve retrieval. Our design therefore uses co-attention only to write queries and never needs it to scan the corpus.

Research Program

The system has three parts.

  1. First-hop retrieval and scoring. A standard first-stage retriever returns candidates. A co-attention cross-encoder initialized from a pretrained encoder rescores them and produces the matrices $A$, $B$ and $s$.
  2. Span selector. A light head over $b_j$ picks spans.
  3. Next-hop retrieval. The selected spans, optionally concatenated with the unresolved question words (those with low total attention to the passage), are sent to the same index.

The loop repeats up to the benchmark's maximum hop count. A small reader then answers from the accumulated passages.

Training is weakly supervised. HotpotQA, MuSiQue and 2WikiMultiHopQA identify which documents are gold at each hop. A span is rewarded when issuing it as a query raises the gold next-hop document's rank. This needs no sentence labels, unlike Baleen. We also run a cheaper distant-label variant, which marks first-hop spans that string-match the gold second-hop title, to see how much the reward-based training adds.

The analysis is organized around three questions:

  • Lexical gap. Does the gain concentrate on questions with low word overlap with the second-hop gold document? We bucket test questions by overlap to check.
  • Error propagation. When the first hop is wrong, how much worse do later hops get? An extracted query can only contain words from a retrieved document. Our hypothesis is that this bounds how far a single first-hop error can drift the search, which a generated query cannot guarantee.
  • Cost. We measure wall-clock time per query against generation loops.

Baselines

Do-nothing reference. Single-shot retrieval with the original question, using both BM25 and Contriever, read by the same reader. It measures how much any iterative scheme buys.

Pseudo-relevance feedback. Classic RM3-style expansion with the top terms of the first-hop documents. This is the naive extractive version of our idea, and the comparison we most need to win: if term statistics pick the bridge as well as attention does, the attention machinery is unjustified. It uses the same indexes, the same hop budget, and expansion size tuned on the development set with the same number of trials as our selector.

IRCoT. The strongest generative competitor, run with the same reader model and the same retrieval index, and with the same maximum number of hops. On HotpotQA, MoPo's paper reports IRCoT at 56.2 downstream F1 (Momentum Posterior Regularization for Multi-hop Dense Retrieval).

Strongest reported retrievers. MoPo reports 82.20 EM@100 retrieval and 64.3 downstream QA F1 on HotpotQA in the same paper. Baleen reports 96.3% answer recall in the top 20 on HotpotQA and over 90% four-hop retrieval at $k=100$ on HoVer. On MuSiQue, the best reported number in the file is L-RAG's accuracy of 0.352. We reproduce Baleen and MoPo on our hardware where code allows; otherwise we compare to these reported figures under matched retrieval depth. GRITHopper (GRITHopper: Decomposition-Free Multi-Hop Dense Retrieval) is included for latency and MoreHopQA. For our method on HoVer and MoreHopQA, we found no reported result on Lacuna for any extractive-query method.

What counts as success. Beating IRCoT's answer F1 at a fraction of its latency is the minimum. Matching MoPo's HotpotQA numbers and beating both on the low-overlap bucket of MuSiQue is the real target.

Ablations isolating the key ingredient. Four ablations isolate what matters:

  • One-directional selection: drop the novelty factor and select by question-to-passage attention alone, which is what one-way late interaction could supply.
  • No sparsity penalty.
  • Random spans of the same length.
  • Gold-bridge oracle: the gold bridge entity is inserted as the query, giving an upper bound.

Why not just…?

Why not just use pseudo-relevance feedback on the first-hop documents? It is the closest simple idea, and it may well be competitive on HotpotQA, where bridge entities are often titles and frequent terms. Our argument is that frequency does not tell you which term the question is missing. In "Inception is a 2010 film directed by Christopher Nolan, starring Leonardo DiCaprio…", feedback has no reason to prefer Nolan over DiCaprio, while the attention from "director" does. The low-overlap MuSiQue bucket and the 3–4 hop HoVer setting settle this. If feedback matches us there, we will report that the attention scorer adds nothing.

Why not just run IRCoT with a strong language model? For Wikipedia questions, a large model often already knows the bridge entity from memory, so IRCoT can be strong. IRCoT's reported HotpotQA F1 of 56.2 trails specialized retrievers. Iterative generation is also slow: L-RAG reports more than 20 seconds per query for ITER-RETGEN on 2WikiMultiHopQA, versus about 2.8 seconds for its non-generative method. GRITHopper reports 1 hour 24 minutes for GPT-4o-based decomposition on MuSiQue. This is a real threat on accuracy, so we split results by whether the reader answers the first hop correctly closed-book. Extraction should win where memory fails, and at minimum should win on cost and on error propagation.

Why not just add a cross-encoder reranker at each hop of an existing pipeline? Reranking improves which passage is chosen. It does not change what the next query says, so the second hop still uses the question's vocabulary. Our ablation that keeps the identical cross-encoder but uses it only to rerank, with the next query formed IRCoT-style or by feedback, measures exactly how much comes from query writing versus scoring.

Why not just use Baleen's condenser? Baleen is the strongest extractive competitor, and on HotpotQA and HoVer it may match us. Two differences are testable. We need no sentence-level labels, so we compare both systems on MuSiQue and 2WikiMultiHopQA trained only with document-level gold. Span queries are also shorter and more targeted than appended sentences, which we test with retrieval recall at equal query length.

Proposed Experiments

Benchmarks. Main benchmarks are HotpotQA (full-Wikipedia setting), MuSiQue-Ans, 2WikiMultiHopQA, and HoVer for longer chains, with MoreHopQA as an out-of-distribution test of chain length.

Metrics. Retrieval recall of all gold documents at $k=20$ and $k=100$, answer F1, and wall-clock latency per query on identical hardware. We add a robustness metric: answer F1 when the first hop is deliberately replaced by the top-ranked wrong document, compared across our method, IRCoT and feedback.

Decisive checks.

  • Lexical-gap check. On questions in the lowest overlap tercile with the second-hop gold document, our method should beat pseudo-relevance feedback and IRCoT on second-hop recall@20. If it does not, the core claim fails.
  • Bidirectionality check. The full scorer must beat the one-directional ablation on that same tercile. Otherwise bidirectional co-attention is not the operative ingredient, and the method reduces to attention-weighted feedback.
  • Error-propagation check. We expect a smaller F1 loss under induced first-hop errors than IRCoT.

Secondary test. As a smaller exploratory test of the same principle outside Wikipedia, we will apply the selector to MISCIPLUS's undermining-context subtask. The question there is whether spans extracted from the "kernel of truth" passage lead to the methodology sections that use different technical vocabulary.