Questions like "In what city was the director of Inception born?" defeat ordinary retrieval-augmented generation, meaning a language model that answers from passages fetched by a search engine. The fact that answers the question, Nolan's birthplace, shares no words with the question. Iterative systems close part of this gap by searching repeatedly. In doing so they pick up a second failure. They do not know which intermediate unknowns they still need, so they stop too early or keep reasoning past the answer, and one wrong intermediate hop spreads into everything after it. This note proposes giving the search loop an explicit ledger of the question's unknowns, built from two sources: a linguistic parse of the question and the language model's own decomposition. Each unknown counts as bound only when retrieved evidence passes cheap symbolic checks on its type and its role in the sentence. The loop stops exactly when the answer slot and every constraint on it are bound.
The approach borrows a lesson from older hybrid language systems. Hand-written rules are precise but brittle, statistical models are robust but sloppy, and the gains come from weighing the two against each other, not from chaining them. A Comparison of Rule-Based and Statistical Methods for Semantic Language Modeling and Confidence Measurement found that interpolating a hand-coded grammar with a maximum-entropy model cut word error by 18.9% relative and improved confidence estimation by more than 20%. Modern multi-hop systems that use linguistic structure are still built as chains. We apply the weighing lesson to the single decision multi-hop loops get most wrong: whether an intermediate answer is actually grounded, and whether the search is finished.
Core Claim
The claim is that a learned arbiter, given both a parse-derived and a model-derived list of unknowns, plus symbolic checks on each candidate binding, will do better than iterative baselines given the same retrieval budget. "Better" here means three things: more accurate stopping decisions, fewer hops spent after the answer is already in hand, and higher exact match on 3–4 hop questions. We expect the advantage to be largest for small models (around 8B parameters), whose own decompositions are least reliable.
The claim makes a falsifiable prediction about where the parse earns its keep. It should help most when the parse and the model's decomposition disagree, and the checks should catch bindings the model accepts wrongly. If a ledger written entirely by the model, with the same checks, matches the hybrid version, then the parser adds nothing. We would report that as the main finding.
How This Differs From What Exists
Four prior lines of work shape the design, each in a different role.
The first supplies the method. Interpretable AMR-Based Question Decomposition for Multi-hop Question Answering (QDAMR) parses questions into Abstract Meaning Representation (AMR), a graph of concepts and their semantic roles. It cuts that graph at unknown nodes to produce sub-questions, reaching 81.17 F1 on the HotpotQA distractor setting and much better sub-question fluency than DecompRC (68.2% vs. 32.6%). We reuse its unknowns-based segmentation to propose slots. QDAMR, however, works only over ten pre-supplied paragraphs, uses a fixed two-hop chain, and has no way to recover from a bad parse. We run in open-corpus retrieval, allow any number of hops, and treat the parse as one vote that can be outvoted.
The second line motivates the termination problem. Leveraging Structured Information for Explainable Multi-hop Question Answering and Reasoning extracts entity–relation triples from retrieved passages and puts them in the prompt. It reports less over-reasoning and higher exact match on 2WikiMultiHopQA, but the triples never steer retrieval or stopping. Think Straight, Stop Smart (TSSS) shows that stopping well pays off: 34.1 exact match on HotpotQA and roughly half the latency of iterative baselines. Its stop rule, though, is an embedding-similarity threshold of 0.85 between consecutive sub-queries. That rule detects repetition, not completion. Our stop rule asks whether every open slot is bound.
The third line is graph-based navigation, which we treat as a competitor, not a component. HopRAG builds a passage graph offline from questions an LLM imagines for each passage, and reaches 42.20 exact match on MuSiQue. It puts structure into the index. We put structure into the query and the stopping decision, and leave the index as plain text, which avoids HopRAG's expensive offline build.
The fourth line constrains the design. Parser errors cascade (QDAMR's own limitation), and the hybrid literature consistently finds that hard rules break on ambiguous language (Function-Based Question Classification for General QA softens its regex rules into weighted logic for exactly this reason). That rules out a design where the parse alone decides what to retrieve or when to stop.
Research Program
The system has three parts.
Building the ledger. Each question is parsed with the off-the-shelf AMR tooling QDAMR used. The unknown nodes become candidate slots, each tagged with its AMR concept (person, city, date, and so on) and its role relative to its predicate (for example, the director as the agent of "direct"). Separately, the LLM writes a decomposition with numbered placeholders in the style of Self-Ask. The two slot sets are aligned by type and by which predicate they attach to. Where they agree, the slot is accepted. Where they disagree, both versions are kept as alternatives.
Iterative retrieval. The loop retrieves for the earliest unbound slot whose dependencies are satisfied. The LLM then proposes a value for that slot from the retrieved text. A proposed value is accepted with probability
$$p(\text{accept}) = \sigma(w_1 c + w_2 t + w_3 r + w_4 a),$$
where:
- $c$ is the model's own confidence in the value (from token log-probabilities),
- $t$ is 1 if a named-entity tagger assigns the value a type consistent with the slot's AMR concept, and 0 otherwise,
- $r$ is 1 if a semantic role labeler finds the value in the expected role of the expected predicate in the supporting sentence,
- $a$ is 1 if the slot came from the agreement of parse and model,
- $\sigma$ is the logistic function.
The weights $w$ are fit on MuSiQue training data. MuSiQue questions are built by composing single-hop questions, so its gold single-hop decompositions give per-slot supervision. A rejected value leaves the slot open and triggers a reformulated search.
Stopping and budgeting. The loop stops when the answer slot and all constraint slots are accepted, or when a fixed retrieval budget runs out. Slot alternatives that were never resolved are spent first under that budget. This gives a simple compute-allocation rule: questions where parse and model disagree get more searches, and questions where they agree get fewer.
Baselines
Floor. The naive reference is single-step retrieval followed by generation. With the same retriever, model, and corpus, it shows how large the multi-hop gap is in our setup. For calibration, Chain-of-Retrieval Augmented Generation reports MuSiQue retriever recall of 29.0% for single-step retrieval versus 47.1% for its iterative chains.
Interleaved reasoning and retrieval. IRCoT uses each chain-of-thought sentence as the next search query. We pair it with ITER-RETGEN, which re-retrieves using whole drafts for two to three rounds. These are the natural competitors for "does an explicit ledger beat free-form iteration." In the TSSS comparison, IRCoT scored 28.0 exact match on HotpotQA.
Strongest efficient structured competitor. TSSS is the closest competitor on the stopping question, because it is the published method that also targets termination and latency. It reports 34.1 exact match on HotpotQA and 9.5 s per MuSiQue sample, against 27.2 s for Iter-RetGen. Beating it means higher exact match together with equal or lower retrieval calls per question.
Strongest accuracy competitors. On MuSiQue this is HopRAG at 42.20 exact match. On 2WikiMultiHopQA it is GRAT, at 75.2% accuracy with Llama3-70B and 66.8% with a fine-tuned Llama3-8B (up from 60.6%). We also include DRAGIN, which reports 0.3931 F1 on 2WikiMultiHopQA with LLaMA2-13B. For HopRAG on HotpotQA and 2WikiMultiHopQA, the file gives only relative gains, so we report no absolute baseline number there.
Fair comparison. All methods are re-run with one fixed retriever over the benchmark's own corpus, the same Llama3-8B and Llama3-70B backbones, and an equal prompt-tuning budget of a fixed number of dev-set trials each. Published numbers are quoted only as sanity anchors, because their settings differ.
Ablations. These isolate the key ingredients. The first is a parser-only ledger, which is QDAMR run as a hard cascade in open retrieval. The second is a model-only ledger that keeps the same type and role checks. The third is the full system with the checks removed, so that $c$ alone decides acceptance.
Why not just…?
Why not just have the LLM write the placeholder plan and skip the parser? This is the most serious threat, and the model-only-ledger ablation tests it directly. There is evidence against model-only plans. Self-Ask scored 25.6 exact match versus 34.1 for structured templates in TSSS, and modular decompositions in QDAMR were judged fluent only 32.6% of the time. But those results come from older or different models. If the model-only ledger with checks matches the hybrid within noise at both model sizes, we conclude the checks matter and the parse does not, and we would rewrite the claim around that.
Why not just stop with a similarity threshold or a fixed number of rounds? TSSS's threshold detects repeated queries, not satisfied needs, and ITER-RETGEN fixes the round count in advance. On MuSiQue the gold hop count is known for every question. So we can measure stopping accuracy directly, and count "post-answer hops" — retrievals made after the gold answer entity was already retrieved — for each stopping rule.
Why not just spend the compute on tree search or best-of-N? GRAT and CoRAG show that accuracy grows with test-time compute, and with enough compute they may win on raw accuracy. Our comparison is therefore matched on retrieval calls and LLM tokens. We plot accuracy against compute for CoRAG-style best-of-N at budgets of 1, 2, 4, and 8 samples. The proposal succeeds only if its point lies above that curve, not merely above its lowest point.
Why not just ask the model "have you answered the question?" Self-verification uses the same model that over-reasons, so its errors are correlated with the errors it is meant to catch. We include it as a stopping-rule variant. The type and role checks are worthwhile only if they reject wrong bindings that self-verification accepts. We measure this by the precision of rejected bindings against MuSiQue gold intermediate answers.
Proposed Experiments
The main evaluation uses MuSiQue-Ans, split by hop count, because it is built to block single-hop shortcuts. It is supplemented by 2WikiMultiHopQA, whose evidence relations let us check role binding, and by HotpotQA in the open setting, not the distractor setting. MultiHop-RAG serves as an out-of-domain news test. Its reported Hits@10 of 0.7467 and GPT-4 accuracy of 0.56 on retrieved chunks (MultiHop-RAG) indicate how much headroom remains.
The metrics are:
- exact match and F1,
- supporting-passage recall,
- retrieval calls and wall-clock time per question,
- stopping accuracy against gold hop count,
- post-answer hops,
- slot-extraction accuracy against MuSiQue's gold decompositions.
Three checks decide whether the idea holds.
First, on 3–4 hop MuSiQue questions with the 8B model, the hybrid must beat both the model-only ledger and TSSS at an equal number of retrieval calls. A tie with the model-only ledger falsifies the parser's role.
Second, the arbiter should behave as a hybrid system is supposed to. Slots where parse and model agree should be accepted with high precision, and the extra searches spent on disagreement slots should account for a measurable share of the accuracy gain. If the gain is spread evenly across questions, the "weigh, don't chain" story is wrong.
Third, at 70B the advantage should shrink. If it grows instead, the mechanism is not compensating for weak model decompositions, as hypothesized, and we would need to explain why.
References
- A Comparison of Rule-Based and Statistical Methods for Semantic Language Modeling and Confidence Measurement
- Interpretable AMR-Based Question Decomposition for Multi-hop Question Answering
- Leveraging Structured Information for Explainable Multi-hop Question Answering and Reasoning
- Think Straight, Stop Smart: Structured Reasoning for Efficient Multi-Hop RAG
- HopRAG: Multi-Hop Reasoning for Logic-Aware Retrieval-Augmented Generation
- Function-Based Question Classification for General QA
- Chain-of-Retrieval Augmented Generation
- Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
- Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy
- GRAT: Guiding Retrieval-Augmented Reasoning through Process Rewards Tree Search
- DRAGIN: Dynamic Retrieval Augmented Generation based on the Real-time Information Needs of Large Language Models
- MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries