Answering "In what city was the director of Inception born?" requires two lookups. The first finds that Christopher Nolan directed Inception. The second searches for Nolan's birthplace. Between the two, the system must pass something forward, and what it passes is the least-studied choice in multi-hop retrieval. Question-decomposition systems pass a short answer string ("Christopher Nolan"). If that string is wrong, every later hop inherits the error. Interleaved systems such as IRCoT pass the accumulated passages and reasoning sentences. That keeps more information but lets irrelevant text pile up until the model drifts off the question. We propose passing a discourse unit instead. This is the clause-sized piece of the retrieved passage that states the bridge fact, together with the chain of central clauses it depends on, and without the side remarks around it. The unit is computed once, when the corpus is indexed, from a rhetorical parse of each passage, so choosing it costs no extra model generation at question time.
The claim is testable and falsifiable. Hold fixed the question plan, the retriever, the language model and the token budget, and vary only what is carried between hops. The discourse unit should beat an answer string, because it keeps enough context to recover from a wrong or ambiguous entity. It should also beat a whole passage, because it discards the text that feeds drift. Most importantly, it should beat an equally long window of text that ignores discourse structure. MuSiQue's hop-by-hop annotations let us see where each version first goes wrong.
Background and the gap
Three bodies of work meet here, and each plays a different role.
Rhetorical Structure Theory (RST) splits text into elementary discourse units (EDUs), which are minimal clauses carrying one proposition. It arranges them in a tree whose links mark each unit as either a nucleus (the central claim) or a satellite (an elaboration, attribution, condition or contrast that supports it). This line of work supplies our method. Beyond Chunking builds RST trees over long documents and retrieves by descending them. It beats both flat chunking and a naive bisection tree, which is evidence that the rhetorical shape, not mere nesting, carries signal. From Context to EDUs (LingoEDU) decomposes context into relation trees whose nodes point back to exact spans of the original text, prunes branches irrelevant to the query, and reports a 14.94% gain on HotpotQA at about 1.2 seconds per document. Disco-RAG parses chunks after retrieval and turns the parse into a writing plan for the generator.
These systems share a limitation. All three apply discourse structure to a single, static query: the tree is used to trim or organize context for one question. None uses it to decide what flows between retrieval steps.
Hierarchical query decomposition supplies the planning layer and also a warning. RoHT builds a tree of sub-questions, routes each to a knowledge base or a text reader, and combines candidate answers from the leaves up. QDAMR splits a question's meaning graph at its "secondary unknowns" (the bridge variables). Is a Question Decomposition Unit All We Need? shows that human-written decompositions correct most model errors. That same paper and Cao et al. document the weakest-link problem: an early wrong answer invalidates the rest of the chain. In every one of these systems, the value passed from one sub-question to the next is an extracted string.
The multi-hop reasoning gap literature both motivates and constrains the design. IRCoT alternates reasoning sentences with retrieval and improves recall by 11–21 points. It also names the failure we target: early irrelevant passages accumulate and the model drifts. GRITHopper argues that generating explicit sub-queries is slow, reporting 6 minutes 42 seconds against 1 hour 24 minutes for GPT-4o decomposition on MuSiQue. That result sets our latency constraint. MuSiQue shows that single-hop shortcuts inflate scores on older benchmarks, which is why MuSiQue is our primary benchmark.
The gap these leave is specific. Discourse structure has been used only to shape context for a fixed query, and decomposition has passed only strings between hops. Nobody has asked what the right unit of hand-off between hops is, or whether rhetorical structure identifies it better than length or generic relevance does.
Core Claim
The state passed between hops should be a nuclear path. For a candidate bridge EDU $e$ (the clause the reader identifies as answering the current sub-question), the nuclear path is $e$ plus every nucleus on the route from $e$ to the root of its paragraph's RST tree, with all satellites dropped. The idea behind this is that satellites carry the side remarks responsible for lexical distractors: attributions, background elaborations, and contrasts that mention other entities. The nuclear ancestors, by contrast, typically name the topic the clause is about. The path therefore keeps what is needed to disambiguate the bridge ("Nolan, the British-American filmmaker…") and drops what pulls retrieval toward distractors.
Because each path is a pointer into a parse computed at index time, the hand-off costs one lookup. Because it keeps its source span, it leaves an auditable trail. And because it does not commit to a single string, we can keep a small beam of alternative paths and let later hops decide between them. This softens the weakest-link failure without the cost of generating a new sub-query at every hop.
Research Program
The system has three stages, and only the middle one is new.
The first stage is planning. A single language-model call turns the question into a decomposition tree in MuSiQue's own format, with sub-questions containing placeholders such as "#1" that refer to the answers of earlier sub-questions. This is RoHT-style planning with no discourse involvement. We keep it deliberately standard so the comparison isolates the hand-off.
The second stage is the hand-off. At each hop, a retriever (BM25, as in IRCoT, plus a dense variant) fetches passages for the current sub-question. A small ranker then scores EDUs inside those passages against the sub-question. We use a LingoEDU-style ranker from From Context to EDUs, with the parse precomputed over the whole corpus. The top $k$ EDUs become candidate bridges. Each candidate's nuclear path fills the placeholder in the next sub-question, so the next retrieval query is the sub-question text plus the path.
The third stage is resolution. At the last hop the reader answers once for each beam branch. Branch scores are combined from the leaves up in the manner of RoHT, and the winning branch's chain of paths is returned as the explanation.
The analysis goes beyond final accuracy. MuSiQue annotates the gold supporting paragraph and intermediate answer for each hop. For every system we can therefore locate the first hop where the gold paragraph falls out of the retrieved set, and measure whether that failure was caused by drift (retrieval pulled toward an entity mentioned only in a satellite) or by commitment (a wrong string was extracted). We will also report the one-time cost of parsing the corpus. The research file notes that parser accuracy is the known bottleneck of discourse methods (Beyond Chunking), so we will measure how sensitive the system is to parse quality by injecting controlled noise into the trees.
Baselines
The naive reference point is one-shot retrieval: the full question is used once as the query, and the reader sees the top passages. This is the setting MuSiQue was built to defeat, and it shows how much any multi-step method adds.
The strongest competitor on this specific question is IRCoT (Interleaving Retrieval with Chain-of-Thought Reasoning). It carries whole passages plus generated reasoning sentences, and its drift failure is exactly the one we claim to fix. The file reports only relative gains for it: up to 15 F1 and 11–21 points of retrieval recall over one-step retrieval across HotpotQA, 2WikiMultihopQA, MuSiQue and IIRC. We found no reported absolute MuSiQue score for IRCoT on Lacuna, so we will rerun it with its released prompts. It will use the same backbone model, the same BM25 index over the same MuSiQue paragraph corpus, and the same cap on passages per step. Its few-shot demonstrations will be tuned on the same development split with the same number of trials we allow ourselves.
The strongest decomposition baseline is a RoHT-style tree that carries answer strings (RoHT). RoHT reports 54.4 exact match on MuSiQue, nearly 13 points above its best baseline. Its setup is not identical to ours, so we treat that figure as a reference and rerun a string-carrying version inside our own pipeline with the same plan and the same beam width. The only difference will then be that each branch carries an extracted string rather than a nuclear path.
The latency reference is GRITHopper (GRITHopper), used with its released model. The file gives its wall-clock time on MuSiQue but no MuSiQue accuracy number (no reported result found on Lacuna for GRITHopper MuSiQue accuracy), so we measure both under identical hardware.
The ablations isolate the key ingredient. At an identical token budget per carried state, we will replace the nuclear path with each of the following:
- the bridge sentence alone;
- a window of the same length centered on the bridge EDU;
- the bridge EDU plus its satellites instead of its nuclei;
- the nuclear path computed on a structure-free right-branching tree, which is the analogue of the bisection control in Beyond Chunking.
The proposal counts as beating the baselines if the nuclear path improves answer F1 and per-hop gold-paragraph recall over both IRCoT and the string-carrying tree on MuSiQue-Ans by a margin that exceeds seed variance. It must also clearly beat the equal-length window and the satellite variant, and keep query-time cost close to the string-carrying tree rather than to per-hop generation.
Why not just…?
Why not just use a bigger model with chain-of-thought? Evidence in the file suggests that process matters more than scale for this problem. IRCoT's 3B model beat a 175B model that retrieved only once. Decomposition alone lifted GPT-3 substantially in Is a Question Decomposition Unit All We Need?. Frontier models also parse document structure poorly: on StructBench they score 43.15% (Claude 4-Sonnet) and 28.63% (o3), against 49.60% for a specialized model. We will still run the strongest available model with linear chain-of-thought and full passages as a ceiling check. If it matches us at comparable cost, the claim fails.
Why not just let the model pick the relevant span itself? This is the real threat. An LLM asked to "quote the sentence that answers #1 plus needed context" might select something as good as the nuclear path. The difference is cost. That selection needs one generation per beam branch per hop, which is exactly the overhead GRITHopper measured as roughly an order of magnitude slower. The nuclear path is a lookup. We will run LLM-selected spans as an additional arm and report accuracy against wall-clock time. If LLM selection wins at matched latency, the contribution shrinks to a cheaper approximation of it, and we will report it as such.
Why not just carry whole passages and rely on long context? Carrying whole passages is roughly what IRCoT does, and its authors describe drift under accumulated context. LingoEDU's gains from pruning (+14.94% on HotpotQA) point the same way. The whole-passage arm of our budget sweep answers this directly: if drift is not a real failure mode on MuSiQue, whole passages will match nuclear paths.
Why not just use a decomposition-free dense retriever? GRITHopper is fast but opaque. MuSiQue's disconnection filtering exists because dense single-hop matching exploits shortcuts, which shows up as a 30-point F1 drop relative to HotpotQA. We compare accuracy, latency and auditability directly. The auditability test checks whether each hop points to a span that a human agrees states the bridge fact. If GRITHopper wins on both accuracy and speed, our case rests only on auditability, and we will say so.
Decisive Checks
We will run the main experiments on MuSiQue-Ans with MuSiQue's gold decompositions available for analysis only, and cross-check on HotpotQA and 2WikiMultihopQA. On HotpotQA, EARnest's bridge-question MAP of 0.74 from Divide & Conquer serves as a retrieval reference point. The metrics are answer exact match and F1, per-hop gold-paragraph recall, beam-recovery rate (the fraction of wrong first-hop answers corrected by a later hop), and wall-clock time.
Two results would decide the question. First, on 3- and 4-hop MuSiQue questions, where drift and commitment errors compound, the nuclear path must beat the equal-length window. If it does not, rhetorical structure adds nothing beyond trimming by length. Second, the error attribution must show that the gain comes from fewer satellite-driven drift errors, not merely from the beam. We test this by running the string-carrying tree with the same beam width. If the gains vanish under a right-branching control tree or under moderate parse noise, the idea does not survive the parser bottleneck, and that negative result is itself worth reporting.
References
- Beyond Chunking: Discourse-Aware Hierarchical Retrieval for Long Document Question Answering
- From Context to EDUs: Faithful and Structured Context Compression via Elementary Discourse Unit Decomposition
- Disco-RAG: Discourse-Aware Retrieval-Augmented Generation
- Reasoning over Hierarchical Question Decomposition Tree for Explainable Question Answering
- Interpretable AMR-Based Question Decomposition for Multi-hop Question Answering
- Is a Question Decomposition Unit All We Need?
- Probabilistic Tree-of-thought Reasoning for Answering Knowledge-intensive Complex Questions
- Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
- GRITHopper: Decomposition-Free Multi-Hop Dense Retrieval
- ♫ MuSiQue: Multihop Questions via Single-hop Question Composition
- Divide & Conquer for Entailment-aware Multi-hop Evidence Retrieval