Introduction
In the field of multi-hop Reading Comprehension (RC), AI systems are required to synthesize information across multiple documents to arrive at a correct answer. While achieving high accuracy is a primary goal, making these systems "explainable" is equally vital for human trust and utility. Traditional approaches rely on extractive methods—simply highlighting existing sentences from the source text. However, these often fail to provide clear, concise reasoning.
This paper introduces SuQA (Summarizer-augmented QA), a framework that shifts the paradigm from extraction to abstractive summarization. By generating new, condensed explanations, SuQA provides justifications that are significantly more compact and human-readable without sacrificing the information necessary to answer complex questions.
The Limitations of Extractive Explanations
Current explainable RC models typically perform extractive sentence selection. While straightforward, this method suffers from three critical flaws:
- Redundancy: Extractive models often include irrelevant details (e.g., middle names, dates, or tangential facts) that clutter the explanation.
- Contextual Dependency: Selected sentences frequently contain unresolved pronouns like "he" or "it," which are incomprehensible when stripped of their surrounding text.
- Incompleteness: The logical "bridge" between two disparate facts is often implied rather than explicitly stated in a single sentence, making simple extraction insufficient for complex reasoning.
The authors argue that an ideal explanation should be concise: minimal in length, sufficient to support the answer, and linguistically comprehensible.
The SuQA Framework
SuQA addresses these issues through a "summarize-then-answer" pipeline consisting of two primary modules:
- Abstractive Explainer (AX): Built on the BART architecture, this module generates a condensed summary of relevant facts instead of merely selecting sentences. It is designed to paraphrase and resolve coreferences to ensure clarity.
- QA Module (QAM): This component receives only the generated summary to produce the final answer. This setup ensures "faithfulness"—if the QAM can successfully answer the question using only the summary, the explanation is proven to be sufficient.
Training via Reinforcement Learning
A significant challenge in developing abstractive explainers is the scarcity of human-annotated datasets. To overcome this, the researchers employed a semi-supervised approach. After initial pre-training on general summarization and a small set of human-labeled explanations (from the R4C dataset), the model was fine-tuned using Reinforcement Learning (RL).
The RL reward function was designed as a geometric mean of three key metrics:
- Conciseness: Encouraging higher compression ratios to strip away "fluff."
- Sufficiency: Rewarding the system when the QA module correctly answers the question based on the summary.
- Comprehensibility: Utilizing a linguistic acceptability scorer to ensure the generated text is natural and grammatically correct.
Key Results and Insights
Evaluation on the HotpotQA dataset revealed that SuQA significantly outperforms extractive baselines in qualitative metrics:
- Superior Compression: SuQA achieved a compression rate 2.9x higher than traditional extractive methods.
- Enhanced Sufficiency: Despite the drastic reduction in word count, human evaluators rated SuQA explanations 2.5 points higher in sufficiency compared to extractive ones.
- Low-Data Efficiency: The RL training regime proved highly effective; with only 298 human-annotated samples, human-judged sufficiency improved by +11.0 points.
Furthermore, the researchers identified a "Disconnected Reasoning" trap in extractive models. Large models often rely on "shortcut" lexical cues in long paragraphs to find answers. By forcing the model to operate on a tiny, abstractive summary, SuQA ensures the system actually utilizes the logical connections required for multi-hop reasoning.
Conclusion
SuQA demonstrates that AI can be a more effective "teacher" by paraphrasing information rather than just quoting it. While there is a slight trade-off in raw F1 scores—likely due to the removal of the aforementioned "shortcuts"—the gains in explanation quality and faithfulness mark a significant step forward for interpretable AI in complex reading tasks. Future work will look toward jointly optimizing the explainer and QA modules to further bridge the performance gap.
Summary updated