Abstract
Multi-hop question answering (QA) requires systems to iteratively retrieve evidence and reason across multiple hops. While recent RAG and agentic methods report strong results, the underlying retrieval--reasoning process is often left implicit, making procedural choices hard to compare across model families. This survey takes the execution procedure as the unit of analysis and introduces a four-axis framework covering (A) overall execution plan, (B) index structure, (C) next-step control (strategies and triggers), and (D) stop/continue criteria. Using this schema, we map representative multi-hop QA systems and synthesize reported ablations and tendencies on standard benchmarks (e.g., HotpotQA, 2WikiMultiHopQA, MuSiQue), highlighting recurring trade-offs among effectiveness, efficiency, and evidence faithfulness. We conclude with open challenges for retrieval--reasoning agents, including structure-aware planning, transferable control policies, and robust stopping under distribution shift.
Introduction
In the landscape of Retrieval-Augmented Generation (RAG), "single-hop" questions (e.g., "What is the capital of France?") are relatively straightforward. However, "multi-hop" questions—such as "What is the population of the city where the 2024 Olympics were held?"—demand a chain of reasoning: first identifying the city, then retrieving its specific population data.
While numerous "agentic" RAG systems have emerged to tackle these complexities, the research community has lacked a unified language to compare them. Most literature focuses on architectural innovations or training objectives, often treating the execution procedure—the logic of when to search, what to retain, and when to terminate—as a secondary implementation detail. This paper argues that these procedural choices are the primary drivers of performance and introduces a standardized framework to make them explicit.
The Four-Axis Design Framework
To categorize and compare diverse multi-hop QA systems, we propose a Four-Axis Design Framework that deconstructs the retrieval-reasoning process into its constituent procedural components.
- Axis A: Overall Execution Plan (The "When"): This axis defines the macro-level sequence. Systems may retrieve all evidence upfront (Retrieve-then-Read), alternate between reasoning and searching (Interleaved), generate a complete roadmap before execution (Plan-then-Execute), or explore multiple reasoning trajectories simultaneously (Test-time Search Scaling).
- Axis B: Index Structure (The "Where"): This refers to how knowledge is organized. Options range from simple document lists (Flat) and hierarchical summaries (Tree) to interconnected entities (Graph/KG) or expansive Long-context windows.
- Axis C: Next-Step Control (The "How"): This axis governs the decision logic for the next action. Decisions can be driven by hard-coded rules, learned policies, or sophisticated search algorithms like Monte Carlo Tree Search (MCTS).
- Axis D: Stop/Continue Criteria (The "Enough"): This determines termination. Systems may stop based on a fixed resource budget, internal model confidence, or the output of a dedicated "verifier" model that checks if the answer is sufficiently supported by evidence.
Benchmarking Multi-hop QA
Our survey analyzes systems across several benchmarks designed to test complex reasoning and evidence integration:
- HotpotQA: Wikipedia-based questions requiring 2-hop reasoning.
- 2WikiMultiHopQA: Combines structured and unstructured evidence with annotated reasoning paths.
- MuSiQue: A high-difficulty benchmark with 2–4 hop questions designed to prevent "shortcuts."
- StrategyQA: Boolean questions requiring implicit multi-hop reasoning.
- FanOutQA: Focused on multi-document aggregation and consistency.
Synthesis of Empirical Trends
By mapping over 100 papers to this framework, we identified several critical trends in the field:
The Shift to Interleaved Execution
Moving from static "Retrieve-then-Read" plans to dynamic "Interleaved" processes consistently improves recall and F1 scores. This is particularly evident in "full-wiki" settings where the necessary evidence is rarely contained in the initial top-k results.
Efficiency vs. Faithfulness
A recurring trade-off exists between system speed and truthfulness. Systems utilizing "verifiers" (Axis D) to validate every reasoning step are significantly more faithful to the retrieved evidence but incur much higher latency and computational cost. Conversely, the rise of "Test-time Scaling"—where systems explore a tree of possible steps—mirrors the reasoning capabilities of advanced models like OpenAI’s o1 but requires substantial search budgets.
Index-Plan Alignment
We observed a frequent "Index-Plan Mismatch." Many systems innovate on the execution plan (Axis A) while continuing to use a basic "flat" index (Axis B). Our analysis suggests that aligning the index structure with the reasoning plan (e.g., using a Graph index for path-based reasoning) represents a significant opportunity for performance optimization.
Challenges and Future Directions
Despite rapid progress, several hurdles remain for retrieval--reasoning agents:
- Generalizability: Control policies are often over-fitted to specific benchmarks and struggle when the "hop depth" or domain changes.
- Scalability: While structured indices (Graphs/Trees) improve accuracy, they are currently more expensive to construct and maintain than flat vector databases.
- Robust Stopping: Most systems rely on rigid budgets. Developing adaptive stopping mechanisms that remain reliable across different LLM scales and noise levels is a critical open challenge.
- Language and Modality: Current research is heavily skewed toward English text, leaving multi-lingual and multi-modal multi-hop QA as vital frontiers for future exploration.
Summary updated