Introduction
Open Book Question Answering (OBQA) requires systems to not only provide an answer but also to retrieve the specific evidence (supporting facts) from a large corpus. Datasets like HotpotQA emphasize this by requiring multi-hop reasoning across multiple documents. Traditionally, this has been addressed using complex, multi-stage pipelines. For instance, one large-scale model might rank passages for relevance, while a second, independent model extracts specific sentences from those passages.
While effective, these decoupled pipelines are computationally expensive and fail to exploit the inherent relationship between passage-level and sentence-level relevance. This paper introduces the "Two-in-One" framework, a single, lightweight transformer model designed to perform both tasks simultaneously, promoting computational efficiency and task synergy.
The Two-in-One Framework
The core of the proposed approach is a single RoBERTa encoder that processes the question and candidate passages in a unified pass.
Shared Representation and Multi-Task Heads
As shown in the architecture diagram, the input is formatted by concatenating the question (Q) with the sentences (S1, S2, ..., Sk) of a passage. Special tokens are strategically placed: the <s> token at the start captures the global context of the passage, while </s> tokens following each sentence capture local sentence-level context. Two Multi-Layer Perceptron (MLP) heads are then applied to these representations: one for passage ranking and another for sentence classification.
Enforcing Logical Constraints
To ensure the model's predictions are logically consistent, the authors introduce two novel constraints:
- Consistency Constraint: This loss function minimizes the difference between the passage relevance score and the maximum relevance score of the sentences within that passage. This enforces the intuition that a relevant passage must contain at least one highly relevant sentence.
- Similarity Constraint: Using a triplet loss, the model is trained to ensure the vector representation of a relevant passage (the
<s>token) is closer to its supporting sentences than to irrelevant ones.
Experimental Setup
The model was evaluated on the HotpotQA Distractor setting. In this benchmark, the system is provided with a question and 10 candidate passages (containing both gold standard passages and "hard" distractors). The goal is to identify the exact supporting sentences required to answer the question.
Results and Discussion
The Two-in-One model demonstrates that joint learning significantly outperforms single-task baselines while remaining competitive with much larger state-of-the-art models.
Performance and Efficiency
As detailed in the results table, the Two-in-One model achieved a 59.17% Exact Match (EM) for supporting facts, a massive improvement over the ~28% EM achieved by single-task baseline models. Furthermore, with an 85.82% F1 score, it performs comparably to the QUARK pipeline (86.97% F1). Crucially, the Two-in-One model achieves this using only ~330M parameters, roughly one-third of the ~1020M parameters required by QUARK. This highlights the model's alignment with "Green AI" principles by reducing the computational footprint without sacrificing accuracy.
Ablation Insights
Ablation studies confirmed the efficacy of the constraints. The consistency constraint successfully reduced the prediction gap between passage and sentence scores from 0.11 to 0.03, proving that the model learned to logically align its multi-level predictions.
Conclusion
The Two-in-One framework proves that joint passage ranking and sentence selection is not only more efficient but also more effective than decoupled pipelines. By leveraging shared representations and logical constraints, the model achieves state-of-the-art performance with a fraction of the parameters.
Future work aims to extend this into a "Three-in-One" model by adding a third head for final answer span extraction and testing the system's performance in full open-domain settings where millions of documents must be searched.
Summary updated