Abstract
Multi-hop question answering requires models to gather information from different parts of a text to answer a question. Most current approaches learn to address this task in an end-to-end way with neural networks, without maintaining an explicit representation of the reasoning process. We propose a method to extract a discrete reasoning chain over the text, which consists of a series of sentences leading to the answer. We then feed the extracted chains to a BERT-based QA model to do final answer prediction. Critically, we do not rely on gold annotated chains or "supporting facts:" at training time, we derive pseudogold reasoning chains using heuristics based on named entity recognition and coreference resolution. Nor do we rely on these annotations at test time, as our model learns to extract chains from raw text alone. We test our approach on two recently proposed large multi-hop question answering datasets: WikiHop and HotpotQA, and achieve state-of-art performance on WikiHop and strong performance on HotpotQA. Our analysis shows the properties of chains that are crucial for high performance: in particular, modeling extraction sequentially is important, as is dealing with each candidate sentence in a context-aware way. Furthermore, human evaluation shows that our extracted chains allow humans to give answers with high confidence, indicating that these are a strong intermediate abstraction for this task.
Introduction
Multi-hop question answering (QA) presents a significant challenge: a model must aggregate disparate pieces of information across multiple documents or paragraphs to arrive at a correct answer. For instance, answering "What government position was held by the woman who portrayed Corliss Archer in the film Kiss and Tell?" requires first identifying the actress (Shirley Temple) and then locating her specific government roles in a separate context.
Traditional end-to-end neural models often act as "black boxes," making it difficult to discern if they are truly reasoning or merely exploiting lexical shortcuts. This lack of interpretability can lead to fragile performance. This paper introduces a framework that explicitly extracts "reasoning chains"—sequences of sentences that logically bridge the gap between the question and the answer—before performing final prediction.
Methodology
The proposed system follows a two-stage pipeline: Chain Extraction and Answer Prediction.
1. Chain Extraction
The goal of the first stage is to identify a sequence of sentences that form a logical path. Rather than treating sentences as an unordered set, the model views them as nodes in a graph where edges represent shared entities or paragraph co-occurrence.
To train this extractor without human-labeled paths, the authors developed "pseudogold" chains using heuristics like Shortest Path (minimizing hops) and Question Overlap. The extractor itself uses BERT-Para, which encodes paragraphs alongside the question to maintain local context. A Recurrent Neural Network (RNN) with a pointer mechanism then sequentially selects sentences to construct the chain.
2. Answer Prediction
The extracted chains—specifically the top-ranked sequences from a beam search—are concatenated and passed to a BERT-based reader. By pruning the vast search space into a concise chain of relevant sentences, the reader can focus on the most critical information, improving both accuracy and computational efficiency.
Experimental Results
The model was evaluated on two prominent multi-hop benchmarks: WikiHop and HotpotQA.
- WikiHop: The model achieved 76.5% accuracy on the test set, establishing a new state-of-the-art at the time. This outperformed complex graph-based architectures, demonstrating the power of sequential chain extraction.
- HotpotQA: Despite not using the dataset's provided "supporting fact" labels during training, the model reached a strong F1 score of 74.11. This highlights the robustness of the heuristic-based training approach.
Key Insights
- Sequentiality is Key: Extracting sentences as an ordered path is significantly more effective than selecting them as an unordered set. Reasoning is fundamentally a directed process.
- Contextual Encoding: Encoding sentences within their paragraph context (BERT-Para) is superior to encoding them in isolation. Local cues, such as coreference, are essential for linking sentences.
- Human Interpretability: Human evaluation confirmed that the extracted chains are highly informative. Participants could answer questions with high confidence using only the model-provided chains, suggesting the model's internal logic aligns with human reasoning.
Conclusion
By forcing a model to produce explicit reasoning chains, we gain both performance and transparency. This approach proves that sophisticated multi-hop reasoning can be learned through simple heuristics and sequential extraction, providing a scalable path toward more interpretable and reliable AI systems.
Summary updated