TL;DR
Large language models (LLMs) struggle to reason effectively over increasingly long contexts, often getting overwhelmed and hallucinating information. This paper introduces ALR², a novel retrieve-then-reason framework that explicitly aligns LLMs with both retrieval and reasoning objectives, significantly improving performance on long-context question answering tasks.
What Problem Does This Paper Address?
As LLMs' context windows grow, their ability to accurately reason over that extended context degrades. This is because LLMs can become overwhelmed by the sheer volume of information, making it difficult to pinpoint and utilize the sparsely distributed relevant evidence needed to answer questions. Simply prompting LLMs to retrieve information often leads to them fabricating ("hallucinating") facts, which then results in flawed reasoning and incorrect answers. The paper aims to solve this by developing a method that guides LLMs to first retrieve relevant information and then reason over it.
Methods
The core of the paper is the ALR² (Aligning LLMs with Retrieval and Reasoning) framework. It's a two-stage process:
- Retrieval: The LLM is tasked with identifying and extracting the most relevant pieces of information from the long context that are crucial for answering a given question.
- Reasoning: The LLM then uses this retrieved, condensed information to formulate the final answer.
The novelty lies in explicitly aligning the LLM's objectives for both retrieval and reasoning. This is achieved through a joint optimization objective that trains the LLM to not only retrieve correct supporting facts but also to generate accurate answers based on those facts. This contrasts with standard Retrieval-Augmented Generation (RAG) approaches, which often treat retrieval and generation as separate or less tightly coupled processes.
Datasets
The paper uses existing datasets adapted for long-context evaluation:
- HotpotQA: A multi-hop question-answering dataset.
- SQuAD: A single-hop question-answering dataset.
- Needle-in-a-Haystack (NIAH) variants: Used for evaluating retrieval performance. These include Single NIAH, Multi-keys NIAH, Multi-values NIAH, and Multi-queries NIAH.
The authors also mention using the RULER benchmark (Hsieh et al., 2024) for evaluation.
Key Insights and Ideas
- Reasoning Degrades Faster Than Retrieval: A key observation is that while LLMs' performance on pure retrieval tasks is relatively stable even with long contexts, their reasoning capabilities degrade significantly as context length increases. This highlights the specific challenge of reasoning over long, complex documents.
- Hallucination is a Major Hurdle: The paper emphasizes that modern LLMs often "hallucinate" retrieved facts when prompted to retrieve information. This fabricated information then poisons the reasoning process, leading to incorrect answers.
- Alignment is Key: The core idea of ALR² is that explicitly training LLMs to optimize for both retrieval accuracy and reasoning correctness is crucial for overcoming long-context challenges. Simply asking an LLM to retrieve and then answer isn't enough; it needs to be guided on how to retrieve effectively for reasoning.
- Generalization Matters: ALR² demonstrates good generalization capabilities, performing well on unseen datasets, suggesting its robustness.
Key Results
- Significant EM Gains: ALR² achieved substantial improvements in Exact Match (EM) scores on long-context QA benchmarks. Specifically, it showed gains of at least 8.4 EM on HotpotQA and 7.9 EM on SQuAD compared to competitive baselines.
- Outperforms Baselines: The method outperformed various prompting techniques (Direct Answering, Retrieve-then-Reason, Quote-First, System-2 Attention) and even fine-tuned models.
- Reduced Hallucinations: ALR² significantly reduced hallucination rates in retrieval, as evidenced by lower hallucination rates and higher recall scores compared to baseline methods.
- Consistent Performance: On HotpotQA, ALR² maintained consistent performance across context lengths from 4K to 128K tokens.
Limitations and Future Work
- Summarization Challenges: Like other RAG-based methods, ALR² may struggle with summarization tasks where all information in the long context is important for the final prediction.
- Granularity: The current approach primarily considers sentence-level retrieval. The authors suggest that allowing users to choose retrieval granularity (e.g., phrase, sentence, or passage) could be a valuable future direction.
Summary updated