Retrieval-augmented generation (RAG) means answering a question by first searching a document collection and then letting a language model read what was found. It works well when one passage holds the answer. It fails on multi-hop questions, where the fact needed for the second search only appears after the first search succeeds. Current iterative systems handle this by writing free-form reasoning between searches, or by filling a fixed template. Neither keeps an explicit record of what kind of fact is still missing, what type of entity should fill each gap, or under what conditions a retrieved fact actually holds. We propose that the model keep a small, typed schema of the evidence it needs: a handful of slots such as "company, acquirer of X, as of 2023," connected by join links. The key change is that retrieved documents are allowed to revise this schema. When a passage shows that the question depends on a time condition, an unexpected intermediate entity, or a finer entity class, the model adds or retypes a slot. The next search is then generated from the unfilled slots.
The payoff should appear on one kind of question: those whose correct decomposition cannot be read off the question text alone. That is the hard core of the multi-hop gap. MuSiQue was built to remove shortcuts, and a single-hop model loses 30 F1 there relative to HotpotQA (MuSiQue). On MultiHop-RAG, temporal and comparison queries are where retrieval pipelines fail most (MultiHop-RAG). The same schema also gives two things for free. It gives a stopping rule: stop once a fully filled, type-consistent join path connects the question's anchor entity to the answer slot. It gives a backtracking signal: when a filled slot violates its type or condition, that hop is re-searched.
Core Claim
A per-question evidence schema that is revised by retrieved text will beat two alternatives on multi-hop questions whose structure is revealed only during retrieval, at matched LLM-call budgets. The first alternative is a schema fixed from the question up front. The second is untyped sub-question decomposition. The claim is falsifiable in a specific way. If the gain does not concentrate on questions where the question-only decomposition differs from the gold decomposition, then schema revision is not doing the work, and the idea reduces to better prompting.
Relation to Prior Work
The closest system is DocSage. It infers a "minimal joinable schema" from the query, extracts tables with error checks, and answers by SQL joins. It reaches 89.2% on MEBench and holds 87.9% with more than 100 entities. That supports our premise that structure beats raw reading on entity-dense questions. DocSage, however, fixes the schema before any evidence is seen, and it assumes uniform tabular rows. Its authors leave open both iterative schema updates and non-tabular chained questions. Our proposal targets exactly those two cases, and it uses a graph of typed slots rather than tables.
A second group builds structure offline. AutoSchemaKG induces a web-scale schema, costs over 78,400 GPU hours, and reports 12–18% multi-hop gains. TRACE-KG jointly induces entity classes and conditional "qualifiers," with provenance, but is evaluated only on single-step subgraph retrieval. We borrow TRACE-KG's idea that qualifiers must be first-class. We do not borrow its heavy multi-pass clustering. Our schema is question-local, has a few slots, and is built at test time.
A third group interleaves retrieval with reasoning over flat text, and supplies our strongest baselines. IRCoT retrieves at each chain-of-thought step. TSSS replaces free-form reasoning with cached templates and stops when successive sub-queries exceed a cosine similarity of 0.85. MEGRAG retrieves triples, then sentences, then passages, and asks an LLM after each hop whether the query is resolved. MEGRAG explicitly lists as limitations that its raw triples omit qualifiers and that it cannot backtrack. REANO builds question-specific graphs, but it relies on REBEL/Wikidata relation types and on one-shot retrieval. AMR-SG filters facts to "active paths" from question to answer, which gives +4.2% on OpenBookQA. That result motivates our path-closure stopping rule. AMR-SG, though, uses fixed semantic parses and a single retrieval batch. PRIMO shows that type constraints cut repetition in multi-hop chains from 55.0% to 12.7%. This constrains our design: slots must carry types. PRIMO, however, uses a static Freebase ontology and no retrieval.
The survey by Retrieval–Reasoning Processes for Multi-hop QA names the "index–plan mismatch," in which sophisticated plans run over flat indexes. We address that mismatch from the plan side, by keeping structure in the plan state instead of rebuilding the index.
Research Program
The system keeps one object per question: a schema made of typed slots, join edges between slots, and optional qualifiers on each edge (time, place, condition).
Initialization. At hop zero, a small backbone (Qwen3-8B, to match MEGRAG) proposes an initial schema from the question alone.
Each hop. The system does four things in order:
- It picks the unfilled slot adjacent to the filled region.
- It writes a search query from that slot's type, the incoming edge, and any qualifiers.
- It retrieves from the shared index.
- It extracts a candidate filler, together with a pointer back to the source sentence.
Revision step. The extractor may also propose a schema edit, of three kinds only:
- Add a slot, when the passage reveals an intermediate entity.
- Refine a slot's type.
- Attach a qualifier, when the passage states a condition under which the fact holds.
Edits are accepted only if they keep a connected path from the anchor to the answer slot. This keeps revision from drifting into the fragmentation that TRACE-KG attributes to schema-free extraction.
Checks and stopping. A filler that contradicts its slot type or qualifier is rejected, and its hop is re-searched with the next-ranked candidate. The system stops when a path is fully filled and consistent, or when the hop budget runs out.
Answering. The final answer is read from the answer slot. The filled path, with its source sentences, is shown as the explanation.
Training. We plan prompting first, then distill traces from a larger model into the 8B backbone, mirroring MEGRAG's distillation so that training budgets are comparable.
Baselines
Naive reference. The do-nothing point is one-shot retrieve-then-read with the same backbone and index. It shows how much any iteration buys. On MultiHop-RAG, the file's reference point for this regime is GPT-4 at 0.56 accuracy with retrieved chunks, against 0.89 with gold evidence, with best retrieval Hits@10 of 0.7467 (MultiHop-RAG). We rerun this regime with our backbone rather than compare across models.
Strongest flat-text iterative baseline: TSSS. TSSS is the strongest published baseline of this kind in the file. It reports HotpotQA EM 34.1, against IRCoT at 28.0 and Self-Ask at 25.6, and MuSiQue latency of 9.5 s (TSSS). We run it on the same 21M-passage Wikipedia FAISS index TSSS used. IRCoT is included as the canonical interleaved method.
Strongest structured iterative baseline: MEGRAG. MEGRAG reports +8.34 F1 on MuSiQue over its strongest baseline and 71.88% fewer LLM calls on HotpotQA (MEGRAG). No absolute MEGRAG F1 on MuSiQue was found on Lacuna, so we reproduce it with its Qwen3-8B setting. We give it the same corpus, backbone, distillation teacher, and a matched cap on LLM calls and tokens per question.
What counts as winning. Beating these baselines means higher F1 on MuSiQue and 2WikiMultiHopQA, and higher accuracy on MultiHop-RAG temporal and comparison queries. It must come at no more LLM calls than MEGRAG, and the difference must be significant under paired bootstrap.
Fixed-schema ablation. This ablation isolates the key ingredient. It freezes the hop-zero schema and forbids edits, which is a DocSage-style up-front schema carried into the iterative loop.
Untyped-slot ablation. This keeps revision but drops types and qualifiers. It is equivalent to dynamic sub-question decomposition.
Stopping-rule swap. This replaces path-closure stopping with the TSSS similarity threshold, and separately with MEGRAG's LLM "is it resolved?" check.
Why not just…?
Why not just use MEGRAG or TSSS with a bigger model? Scale helps the reader but not the search. MultiHop-RAG shows GPT-4 losing a third of its accuracy purely from missed evidence. IRCoT shows a 3B model with iterative retrieval beating 175B one-shot retrieval (IRCoT). This remains a live threat on HotpotQA, where MEGRAG reports that nearly 90% of questions resolve in one hop. We therefore expect no gain there, and we weight conclusions on MuSiQue and MultiHop-RAG.
Why not just induce the schema up front, as DocSage does? DocSage's results show that up-front schemas work when the needed attributes are predictable from the query. This is the most serious alternative. The fixed-schema ablation tests it directly. If the fixed schema matches the revisable one even on questions whose gold decomposition differs from the question-only one, we drop the revision claim. What would remain is a smaller contribution about typed stopping.
Why not just decompose into sub-questions with variables? A decomposition is a schema without types or conditions. PRIMO's repetition results and MEGRAG's own admission about missing qualifiers suggest that untyped chains accept wrong fillers silently. The untyped-slot ablation measures exactly this. We will report how often type or qualifier rejection triggers a re-search, and what fraction of those re-searches fix the answer.
Why not just build an offline knowledge graph, as AutoSchemaKG does? That approach costs tens of thousands of GPU hours and still inherits web-scale biases. Our method runs per question over the existing passage index. We report wall-clock time next to TSSS's 9.5 s MuSiQue figure, so the cost comparison is explicit.
Proposed Experiments
Main comparison. We compare on MuSiQue-Ans, 2WikiMultiHopQA, and MultiHop-RAG, with HotpotQA reported for comparability. We report answer F1 or EM, supporting-fact recall, LLM calls, tokens, and latency.
Decisive check. The decisive analysis is stratified. For each MuSiQue question, we compare the backbone's question-only decomposition against MuSiQue's gold single-hop decomposition, and split questions into "predictable" and "revealed-structure" groups. The prediction is that the revisable schema's advantage over the fixed-schema ablation is near zero on the predictable group and clearly positive on the revealed-structure group. On MultiHop-RAG, we expect the gain to sit in the temporal and comparison categories, where qualifiers matter, and not in inference queries.
Stopping rule. We measure premature-stop and over-search rates for path closure against the two alternative stopping rules. Both rates are computed against gold hop counts.
Failure analysis. We audit 200 revised schemas by hand for spurious edits. If more than a small fraction of accepted edits are wrong, revision is adding noise rather than signal, as TRACE-KG warns for uncanonicalized structure, and we would tighten the acceptance rule before claiming the result.
References
- ♫ MuSiQue: Multihop Questions via Single-hop Question Composition
- MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries
- DocSage: An Information Structuring Agent for Multi-Doc Multi-Entity Question Answering
- AutoSchemaKG: Autonomous Knowledge Graph Construction through Dynamic Schema Induction from Web-Scale Corpora
- Beyond Predefined Schemas: TRACE-KG for Context-Enriched Knowledge Graph Generation
- Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
- Think Straight, Stop Smart: Structured Reasoning for Efficient Multi-Hop RAG
- MEGRAG: Multi-Granular Evidence Graphs for Answer-Aware Multi-Hop RAG
- REANO: Optimising Retrieval-Augmented Reader Models through Knowledge Graph Generation
- Dynamic Semantic Graph Construction and Reasoning for Explainable Multi-hop Science Question Answering
- PRIMO: Progressive Induction for Multi-hop Open Rule Generation
- Retrieval--Reasoning Processes for Multi-hop Question Answering: A Four-Axis Design Framework and Empirical Trends