When a retrieval-augmented language model (a model that searches a document collection and reads the results before answering) handles a question needing several facts, it usually writes one linear chain of thought and searches along the way. If the answer comes out right, we rarely check whether each fact it relied on was actually found in a retrieved document or was guessed from the model's own memory. This note proposes a different arrangement. The model first writes its reasoning as a small dependency graph: each factual premise is a node that must be backed by a retrieved sentence, each deduction is a node that must follow from the nodes it points to, and any fact the model supplies from memory is labeled as such. When a check fails, the system re-searches only the failing node and recomputes only the nodes that depend on it, rather than rerunning the whole pipeline.
This matters because the metrics currently used for multi-hop retrieval mostly reward the final answer. On competition math, DAG-Math showed that answer accuracy can greatly overstate sound reasoning. Gemini-2.5-Flash reached 52.4% pass@1 on AIME 2025 but only a 17.0% "perfect reasoning rate", the fraction of answers whose derivation graph is fully connected and justified. Nobody has measured the retrieval analog of that gap: answers that are correct but not actually supported by what was retrieved. We propose to measure it, to build a system that targets it, and to test whether dependency structure, rather than per-step verification alone, is what closes it.
Core Claim
The claim has three parts. First, on multi-hop benchmarks with gold supporting evidence, current retrieval pipelines have a large gap between answer accuracy and grounded answer accuracy. Second, treating each premise node in a reasoning graph as a retrieval obligation, checked by entailment against a retrieved sentence, closes much of that gap. Third, because the graph records which conclusions depend on which premises, a failed obligation can be repaired locally: re-retrieve that node, then recompute its descendants. Local repair should recover most of the benefit of a full rerun at a fraction of the cost.
We will call a question grounded-correct when two conditions hold. The answer must be correct, and every premise node the answer depends on must be entailed by a gold supporting passage. We then define the grounded answer rate as
$$\mathrm{GAR} = \frac{1}{N}\sum_{i=1}^{N} \mathbb{1}[\text{answer}_i \text{ correct}] \cdot \prod_{k \in K_i} \mathbb{1}[\text{node } k \text{ entailed by a gold passage}],$$
where $N$ is the number of questions and $K_i$ is the set of retrieved-premise nodes that question $i$'s answer depends on. Nodes the model explicitly labels as coming from memory are reported separately rather than silently counted as grounded.
How This Differs From the Closest Work
The closest work comes from three lines. The first grounds dependency graphs, but in the wrong place or at the wrong time. GRiD splits reasoning graphs into knowledge-extraction nodes and reasoning nodes that cite their parents. Its knowledge nodes draw only from model weights, and it hits a hard ceiling on GPQA Diamond when those weights lack the fact. We reuse its node split but make knowledge nodes external retrieval obligations, keeping a memory-sourced node type only as an explicit, counted fallback. CGRiC builds claim-dependency graphs over cited outputs and repairs only the riskiest claim, preserving 78% of tokens at 35–47% overhead. However, it certifies a finished answer after generation. We move the same repair idea inside the retrieval loop, where a repaired premise changes what gets retrieved and deduced downstream.
The second line imposes structure on multi-hop retrieval but not arbitrary dependency structure. ChainRAG substitutes each hop's answer into the next sub-question to avoid vague references. It is strictly linear, and errors propagate forward. TSSS uses a fixed linear template with a similarity-based stop rule. DTKG routes each question into either a parallel track or a chained track, and so fails on hybrid questions that need both. Our graph permits joins and branches. Topology is not the contribution by itself, though; it is what makes local repair well-defined.
The third line supplies components and constraints. PropRAG motivates retrieving atomic propositions rather than passages, and we use proposition-level units. It also finds paths by embedding similarity without ever checking that one proposition supports the next, which is exactly the check we add. FlowScore constrains our design. Its step-level judges falsely flagged valid steps whenever an unstated commonsense premise was involved, and it needed one model call per step. For that reason we allow explicit memory-sourced bridge nodes rather than forcing every premise to be retrieved, and we use a small entailment model for most checks.
Research Program
The system has four stages.
In the first stage, a planner prompt asks the backbone model to emit a dependency graph for the question. Each node has an identifier, a type (retrieved premise, memory premise, or deduction), a natural-language statement with placeholders for values not yet known, and a list of parent identifiers. The format follows the explicit-parents templates of DAG-Math and GRiD.
In the second stage, the executor processes nodes in dependency order. For a retrieved premise, it fills placeholders from the answers of its parents, which carries ChainRAG's rewriting over to joins. It then retrieves the top candidate propositions with a dense retriever over a proposition index. An entailment check accepts the node only if some retrieved proposition entails the filled statement. For a deduction node, the executor reads only its parents' statements and checks that they entail the conclusion. Memory premises pass without a retrieval check but are logged.
In the third stage, failure triggers repair. A failed retrieval check triggers up to two re-retrievals with query reformulations for that node alone. A failed deduction check triggers regeneration of that node from its parents. After any repair, only descendants are recomputed. If repair budgets run out, the system either abstains or answers with the failed node marked.
In the fourth stage, the final answer is read from the sink node.
Around this system we build the measurement: GAR on HotpotQA, MuSiQue, and 2WikiMultiHopQA. All three provide gold supporting evidence, and MuSiQue and 2WikiMultiHopQA provide explicit hop structure. This lets us score node grounding without human annotation. We will hand-audit 200 graphs to estimate how noisy the entailment judge and the graph parser are, following CGRiC's practice of treating extraction noise as a measured quantity rather than assuming it away.
Baselines
The naive reference point is single-shot retrieve-then-read. The backbone retrieves the top passages once with the same dense retriever and answers directly. It anchors how much any multi-step method adds and gives the GAR floor. We also run closed-book answering, since ArgRAG found that naive retrieval can fall below no retrieval on noisy corpora.
The strongest linear interleaved baselines are IRCoT and Self-Ask, plus TSSS as the best structured linear method. For reference, TSSS reports HotpotQA exact match of 34.1, against 28.0 for IRCoT and 25.6 for Self-Ask. It also reports 9.5 seconds per MuSiQue sample, against 27.2 for Iter-RetGen. These are the methods a practitioner would actually deploy, and they already interleave reasoning with search.
The strongest graph-retrieval baseline is PropRAG. It reports 53.9 F1 on MuSiQue, compared with 38.5 for GraphRAG, and a 64.9 average zero-shot F1 across five datasets, compared with 62.9 for HippoRAG 2. It is the best method in the file on MuSiQue F1 under a comparable open-domain setting, and it shares our proposition index. DTKG's 83.0% on MuSiQue uses a semantic-match accuracy metric and a knowledge-graph backend, so we cite it but do not treat it as directly comparable. ChainRAG reports only a relative gain of about 60% F1 over naive retrieval; no absolute reported number was found on Lacuna for ChainRAG's MuSiQue F1.
For fairness, every method is re-run with the same backbone LLM, the same retriever and proposition index built from the same corpus, and the same tuning budget of prompt variants chosen on a shared development split. Each method is capped at the same maximum number of model calls per question. Published numbers are quoted only as sanity checks on our reimplementations, not as comparison points.
Two ablations isolate the key ingredients. The first, graph without checks, keeps the planner and dependency-order execution but removes entailment checks and repair. It tests whether decomposition alone explains any gain. The second, checks with full rerun, keeps verification but restarts the whole pipeline on any failure. It isolates the value of local repair.
Beating the baselines means the following. GAR must be higher than PropRAG and TSSS by a margin beyond the judge-noise interval estimated from the hand audit, while answer F1 is no lower than the best baseline's. Local repair must reach at least most of the full-rerun method's GAR with clearly fewer model calls.
Why not just…?
Why not just add a per-step verifier to a linear chain like IRCoT? This is the most serious threat, because verification rather than graph topology might carry the whole effect. We will run exactly this system, linear IRCoT with the same entailment checks and a retry on failure, as a baseline. We will then stratify results by MuSiQue's reasoning-graph shapes. If topology matters, the advantage should sit on shapes with joins or parallel branches. There, a linear chain must serialize independent premises, and a retry after a late failure re-derives unrelated earlier steps. If linear-plus-verifier matches us on those shapes, we will conclude that the dependency graph is bookkeeping, not mechanism, and report that.
Why not just decompose into a sub-question graph and skip verification? Plan-then-execute decomposition is already a standard design axis in the four-axis multi-hop survey, and it may match us on answer F1. That is why answer F1 is not our primary success criterion. The graph-without-checks ablation tests directly whether decomposition alone raises GAR. We expect it to leave many correct-but-ungrounded answers, the pattern MuSiQue's construction was designed to expose in MuSiQue through its roughly 30-point F1 drop for single-hop shortcut models.
Why not just sample several full runs and take a majority vote? Self-consistency costs roughly 400% overhead according to CGRiC. FlowScore reports that sampling-based detection reached only about 25% average precision on HumanEval, because consistent errors survive voting. It also does nothing to certify grounding. We include best-of-five voting over the TSSS pipeline at matched call budget to confirm this on GAR.
Why not just put all retrieved passages into a long-context model? DualResearch found that compressing a 41.5k-token log into a 4.2k-token graph improved GPQA accuracy by 6.06%. That evidence suggests long raw context hurts, but it comes from a different setting. We therefore run a top-50-passage long-context baseline on MuSiQue and score both its answer F1 and its GAR, using its cited sentences as premises.
Proposed Experiments and Decisive Checks
The first experiment establishes the gap. We report answer F1 and GAR for every baseline on HotpotQA distractor and full-wiki, MuSiQue-Ans, and 2WikiMultiHopQA. If GAR tracks answer F1 closely for strong baselines, the motivating premise fails and the project narrows to repair efficiency.
The second experiment is the main comparison, stratified by hop count and MuSiQue graph shape, with the two ablations and the linear-plus-verifier baseline.
The third experiment is a controlled failure-injection test. For each question we remove the gold proposition for one randomly chosen hop from the index, then restore it at repair time under an alternate phrasing. We measure GAR recovered per extra model call under three strategies: local repair, full rerun, and linear retry. The prediction that decides the repair claim is this: local repair recovers most of full rerun's GAR at a cost that grows with the size of the failing node's descendant set, not with the whole graph.
The fourth experiment is a memory-node audit. It reports how often the planner labels premises as memory-sourced, and how often those premises are actually present in gold evidence, which would indicate lazy labeling. It also reports how often labeling them as retrieved would have been falsely rejected by the judge, which is the implicit-premise problem FlowScore raised.
All latency and token costs are reported alongside the MuSiQue per-sample times from TSSS, so that grounding gains can be weighed against the cost of iterative retrieval.
References
- DAG-Math: Graph-of-Thought Guided Mathematical Reasoning in LLMs
- Dependency Matters: Enhancing LLM Reasoning with Explicit Knowledge Grounding
- CGRiC: Compositional Risk Certification for Structured LLM Outputs
- Mitigating Lost-in-Retrieval Problems in Retrieval Augmented Multi-Hop Question Answering
- Think Straight, Stop Smart: Structured Reasoning for Efficient Multi-Hop RAG
- DTKG: Dual-Track Knowledge Graph-Verified Reasoning Framework for Multi-Hop QA
- PropRAG: Guiding Retrieval with Beam Search over Proposition Paths
- Hallucination Detection from Structural Reasoning Model
- ArgRAG: Explainable Retrieval Augmented Generation using Quantitative Bipolar Argumentation
- Retrieval--Reasoning Processes for Multi-hop Question Answering: A Four-Axis Design Framework and Empirical Trends
- ♫ MuSiQue: Multihop Questions via Single-hop Question Composition
- DualResearch: Entropy-Gated Dual-Graph Retrieval for Answer Reconstruction