Abstract
Complex Question Answering (CQA) often requires multi-hop reasoning across multiple documents. Current Retrieval-Augmented Generation (RAG) systems frequently fail due to the "bounded-recall" problem, where missing a single piece of evidence breaks the reasoning chain. We present SUNAR, a framework that integrates neighborhood-aware graph exploration with LLM-based semantic uncertainty estimates. SUNAR achieves significant performance improvements, notably a 31.84% boost over state-of-the-art methods on challenging benchmarks.
Introduction
The "Retrieve-and-Reason" pipeline is the standard for modern QA. However, in Complex QA (CQA), the reasoning stage is limited by the quality of the initial retrieval. If a retriever fails to surface even one necessary document for a multi-hop query, the Large Language Model (LLM) cannot recover the answer. This "bounded-recall" problem is the primary bottleneck in CQA.
Methodology: The SUNAR Framework
SUNAR (Semantic Uncertainty based Neighborhood Aware Retrieval) addresses retrieval limitations through three core innovations:
1. Neighborhood Aware Retrieval (NAR)
NAR leverages the "clustering hypothesis"—the idea that semantically similar documents are often relevant to the same queries. SUNAR constructs an offline document-neighborhood graph. During retrieval, it iteratively explores the neighbors of high-scoring documents, bridging the gap when a document's text doesn't perfectly match the query but is semantically related to other relevant evidence.
2. Answer Semantic Uncertainty (ASU)
To filter out distractors, SUNAR uses the LLM to generate multiple candidate answers for a sub-query based on a specific document. High semantic variance (uncertainty) among these answers suggests the document is noisy or irrelevant. Conversely, consistent answers signal a high-quality document.
3. Meta Evidence Reasoner (MER)
MER serves as a post-hoc correction layer, analyzing the entire reasoning path and all gathered evidence simultaneously to mitigate cascading errors from earlier steps.
Experimental Evaluation
SUNAR was evaluated on two multi-hop QA benchmarks: MuSiQue (MQA) and 2WikiMultiHopQA (WQA), using a corpus of over 560,000 documents.
Key Results
SUNAR consistently outperformed established baselines such as Self-Ask, ReAct, and Searchain. On the MuSiQue dataset, SUNAR achieved a score of 32.75 (cover-EM), a substantial improvement over the 24.84 achieved by Self-Ask.
The framework also demonstrated significant recall improvements. On MQA, SUNAR's Recall@10 reached 0.459, nearly doubling the 0.240 achieved by standard retrieval methods. Furthermore, the approach proved LLM-agnostic, showing gains across models like GPT-4o-mini, Llama 3.1, and Mistral v0.2.
Discussion and Limitations
While SUNAR provides a robust solution to the recall gap, it introduces higher computational costs due to the multiple LLM calls required for the ASU component. Additionally, the offline neighborhood graph requires updates as the corpus evolves. Future research may focus on more efficient uncertainty estimation techniques, such as conformal prediction, to reduce latency.
Conclusion
SUNAR demonstrates that improving retrieval recall at small depths is more impactful for CQA than simply scaling model size. By combining graph-based proximity with semantic uncertainty, SUNAR effectively overcomes the bounded-recall problem in multi-hop reasoning.
Summary updated