Bridge Anchors: Closing the Lexical Gaps That Chunking Creates in Multi-Hop Retrieval

Multi-hop question answering requires finding a first piece of evidence, reading off an intermediate "bridge" entity, and then searching for a second piece of evidence that mentions that entity. Retrieval systems split documents into chunks before indexing them, and that split quietly breaks the second step. A chunk deep inside a long report often refers to its subject as "the company," "it," "the NTSB," or an abbreviation defined three pages earlier. The bridge entity's name is not in that chunk, so neither keyword search nor most embeddings can connect the hop-2 query to it. This is a vocabulary mismatch, but chunking produced it; the user's phrasing did not. Current systems repair it at query time, either by having an LLM rewrite every sub-query or by stuffing ever-larger contexts into a long-context reader. Both approaches are expensive, and both still fail when the needed chunk never surfaces.

We propose to fix the problem where it arises, at indexing time. A cheap hybrid extractor runs over each document before it is chunked. Deterministic rules catch rigid alias patterns (titles, section and table headers, parenthetical abbreviations, "hereafter X," "known as Y"), and a neural coreference model catches pronouns and descriptions such as "the firm." The extractor attaches to every chunk a small set of canonical entity names that the chunk refers to but may not spell out. We call these bridge anchors. At query time, the bridge entity extracted from hop-k evidence is joined exactly against the anchor index to propose hop-(k+1) candidates, in parallel with ordinary dense retrieval. The transition between hops becomes a symbolic lookup and no longer depends on the LLM guessing the right rewrite.

Core Claim

Our claim has two parts. The first is diagnostic: on long-document multi-hop QA, a substantial fraction of hop-2 retrieval failures are segmentation-induced lexical gaps, where the gold chunk lacks any surface form of the bridge entity although the entity appears elsewhere in the same document. Nobody has measured this. The second part is about the method: propagating entity identity across chunk boundaries at index time, with rules for precision and a neural model for recall, recovers most of those failures. It does so at a fraction of the cost of query-time LLM rewriting or full LLM decontextualization of every chunk, and it should help an iterative retriever like IRCoT rather than replace it.

The closest existing work frames the difference. Hierarchical Lexical Graph builds an entity-relationship tier by decomposing text into LLM-extracted atomic propositions. That decomposition cuts propositions loose from their surrounding discourse, and the file notes it relies on expensive multi-stage LLM calls and lacks rule-based verification. We keep chunks intact and only annotate them. ChainRAG identified the "pronoun trap" on the query side, reporting up to an 18% Recall@2 drop when sub-queries say "this author," and fixed it with an iterative retrieve-answer-rewrite loop. We address the mirror-image problem on the document side: the passage says "he," not the name. Persistent context headers, which prepend the opening of a document to every chunk as in Zhang et al.'s summarization work, resolve only the document's main subject. Anchors cover secondary entities and locally defined aliases.

Research Program

The work has three parts.

The diagnostic. For each benchmark question, we locate the gold hop-2 evidence chunk under a standard 512-token chunking. We then label whether it contains any surface form of the bridge entity, distinguishing an exact string, a known alias, or neither. Retrieval failures of a strong iterative retriever are cross-tabulated against this label. If "neither" chunks do not account for a disproportionate share of failures, the method has no target, and we would report that as a negative result.

The anchor extractor. The extractor follows the dual-path division of labor that hybrid IE has validated. Rules handle rigid, high-precision patterns, as in the email and URL extraction of Cohen et al. and OCR++, while a statistical model handles variable language. Concretely, the rules come in three kinds:

  • Abbreviation and alias definitions, which create alias-to-canonical mappings that are valid for the rest of the document.
  • Document titles, section headings and table row/column headers, which become anchors for every chunk under their scope.
  • Type checks, which reject a coreference link when, for example, a person pronoun resolves to an organization.

A coreference model runs over overlapping windows so that chains crossing chunk boundaries are not cut. Overlapping windows are the standard boundary-effect remedy used in InstructCoder and the Chilean dictatorship knowledge-graph work. We use overlap only during extraction, not in the index, so that retrieval does not pay the redundancy cost. Each chunk then stores its anchor set as a separate sparse field, along with a flag for whether each anchor came from a rule or from the model.

Retrieval with anchors. At hop k, the reader names the bridge entity, as IRCoT-style reasoning sentences already do. That name and its rule-derived aliases are looked up exactly in the anchor field. The hits are merged with dense hits for the hop-(k+1) query and reranked with a cross-encoder. Because this is a drop-in candidate generator, it can sit inside IRCoT or CoRAG without changing their reasoning loops. Indexing cost is one coreference pass plus regexes per document, with no LLM calls.

Baselines

The naive reference point is single-pass dense retrieval over disjoint 512-token chunks with an LLM reader. On MuSiQue, CoRAG reports single-step retriever recall of 29.0%, against 47.1% for its own chained retriever. We expect our long-document variant to be lower.

The strongest practical competitor is IRCoT, because it is the standard iterative loop our method plugs into and it already surfaces bridge terms through its reasoning. Its paper reports gains of 11–21 recall points and up to 15 F1 over one-step retrieval, but in this file no absolute MuSiQue F1 for IRCoT is reported on Lacuna. We therefore rerun it ourselves with the same retriever, chunking, reader model, prompt, and number of hops as our variant. The only difference between the two systems is the extra anchor candidate stream. Beating IRCoT means a statistically significant gain in hop-2 gold-chunk Recall@5 and in answer F1 on the long-document splits, with no loss on the original paragraph-level splits.

ChainRAG is the strongest query-side lexical-gap fix. It reports roughly 60% average F1 improvement over NaiveRAG, but no absolute MuSiQue number for it is found on Lacuna. We rerun it with the same reader, and also run ChainRAG plus anchors to test whether the two fixes are complementary.

LongRAG is the "fewer boundaries" competitor. It reports 64.3% EM zero-shot on HotpotQA and 25.9 F1 versus 22.5 for traditional RAG on Qasper (LongRAG). We compare against it at a matched total reader-token budget.

Three ablations isolate which ingredient matters:

  • Title/context-header only, which tests whether main-subject propagation explains the gain.
  • Rules only and coreference only, which test whether the hybrid split matters.
  • LLM decontextualization, an upper-bound-style competitor that rewrites every chunk to be self-contained with an LLM, compared on both accuracy and indexing cost.

Why not just…?

Why not just prepend the document title to every chunk? This is the most serious threat. On Wikipedia paragraphs, the title usually is the bridge entity. We expect the title header to capture most of the gain on paragraph-level MuSiQue and HotpotQA. That is why the main evaluation uses long, multi-entity documents (full pages, SEC filings, NTSB reports, papers), where hop-2 evidence often concerns a secondary entity or an abbreviation. The diagnostic will report what fraction of "neither" cases a title would resolve. If that fraction exceeds about 80% on the long-document sets, we will narrow the claim to an alias-and-header method and say so.

Why not just use bigger chunks or a long-context reader? Fewer boundaries means fewer severed references, and LongRAG shows real gains. But LongRAG itself reports degradation past roughly 30K tokens. MuLD shows long-range narrative reasoning remains unsolved even with native long attention. GPT-4 applied directly to whole manuscripts reached only 0.28 recall on MPEA (ad-hoc scientific IE study). The matched-budget comparison will settle this. If 4K-token units at equal reader cost match anchors, then anchors are not worth it.

Why not just let the LLM rewrite each hop's query, or decontextualize each chunk? Query rewriting cannot add the bridge name to a passage that lacks it. Dense similarity between "Felix Salten's birthplace" and "he was born in Pest" depends on surrounding context that the chunk may not contain. Chunk decontextualization does fix this, but at one LLM call per chunk over the whole corpus, and with the drift risk that the file notes for generative expansion. We report the accuracy versus indexing-cost curve. If decontextualization wins by a wide margin at affordable cost, the rule component is not earning its place.

Proposed Experiments

The primary testbeds are long-document versions of multi-hop QA. MuLD's modified HotpotQA already uses full pages above 10,000 tokens. For MuSiQue, we re-embed each gold paragraph in its source Wikipedia article; this is a construction step we must validate. We also use the five HLG corpora, especially SEC-10Q and NTSB, where HLG reports 73.6% average correctness for its best variant. Paragraph-level MuSiQue and HotpotQA serve as controls where we expect small gains. Retrieval for MuSiQue is a known hard case: the best Select+Answer model reaches 47.3 F1 against 78.0 for humans (MuSiQue). HybridQA tests whether header anchors repair the cell-to-passage linking failures that drove HYBRIDER's 43.8% EM despite roughly 88% component accuracy (HybridQA).

We report the following metrics:

  • hop-2 gold-chunk Recall@k, broken down by the diagnostic label;
  • answer EM and F1;
  • MuSiQue's disconnected-reasoning score, to confirm that gains come from connected retrieval rather than shortcuts;
  • anchor precision on a hand-audited sample;
  • indexing compute and LLM calls per question.

Four checks are decisive. First, the diagnostic must show that "neither" chunks are a large share of IRCoT's hop-2 failures. Second, anchors must recover a majority of those failures while leaving failures in the other two labels unchanged; this specificity rules out a generic recall boost. Third, anchors must beat the title-header ablation on the long-document sets. Fourth, IRCoT plus anchors must match or beat LLM decontextualization at a small fraction of its indexing cost. If any of these fails, the corresponding part of the claim is dropped.