Learning to Explain: Datasets and Models for Identifying Valid Reasoning Chains in Multihop Question-Answering

Harsh Jhamtani, Peter Clark

Abstract: Despite the rapid progress in multihop question-answering (QA), models still have trouble explaining why an answer is correct, with limited explanation training data available to learn from. To address this, we introduce three explanation datasets in which explanations formed from corpus facts are annotated. Our first dataset, eQASC, contains over 98K explanation annotations for the multihop question answering dataset QASC, and is the first that annotates multiple candidate explanations for each answer. The second dataset eQASC-perturbed is constructed by crowd-sourcing perturbations (while preserving their validity) of a subset of explanations in QASC, to test consistency and generalization of explanation prediction models. The third dataset eOBQA is constructed by adding explanation annotations to the OBQA dataset to test generalization of models trained on eQASC. We show that this data can be used to significantly improve explanation quality (+14% absolute F1 over a strong retrieval baseline) using a BERT-based classifier, but still behind the upper bound, offering a new challenge for future research. We also explore a delexicalized chain representation in which repeated noun phrases are replaced by variables, thus turning them into generalized reasoning chains (for example: "X is a Y" AND "Y has Z" IMPLIES "X has Z"). We find that generalized chains maintain performance while also being more robust to certain perturbations.

Introduction

Modern AI models have achieved impressive performance on multihop question-answering (QA) tasks, yet they often struggle to provide faithful explanations for their answers. A significant challenge in this domain is "non-evident" reasoning, where the necessary logical steps involve facts not explicitly mentioned in the question. For example, answering whether a suit of armor conducts electricity requires the intermediate knowledge that armor is typically made of metal.

Existing datasets often have reasoning structures implicitly tied to the question phrasing. To address this, we introduce new datasets and modeling techniques focused on identifying valid reasoning chains from a corpus of facts. These chains must connect the question to the answer through a series of logical steps, as illustrated in the figure below.

Examples of valid and invalid reasoning chains associated with correct and incorrect answer options.

Methodology

Our approach involves two primary stages: candidate chain construction and chain scoring. We utilize a retrieval engine to identify pairs of facts that potentially link a question to an answer, prioritizing word overlap to ensure logical "chaining." These candidate chains are then evaluated using a BERT-based classifier trained to distinguish between valid and invalid reasoning paths.

Generalized Reasoning Chains (GRC)

A key innovation in our work is the use of Generalized Reasoning Chains. We "delexicalize" reasoning chains by replacing recurring noun phrases with variables (e.g., X, Y, Z). This process transforms specific factual statements into abstract logical patterns. By training models on these delexicalized representations, we encourage them to learn the underlying structure of a valid argument rather than relying on specific lexical associations, which can often lead to overfitting or fragility.

Transformation of reasoning chains into delexicalized Generalized Reasoning Chain (GRC) representations.

Datasets

We introduce three new benchmarks to facilitate research in explainable multihop QA:

  • eQASC: A large-scale dataset containing over 98,000 annotations for the QASC dataset. It is the first to provide multiple candidate explanations (both valid and invalid) for each answer.
  • eQASC-perturbed: A dataset designed to test model robustness. It contains valid chains that have undergone "immaterial" perturbations (e.g., synonym replacement) by crowd-workers to see if the model’s judgment remains consistent.
  • eOBQA: An out-of-domain evaluation set based on OpenBookQA, used to test the generalization capabilities of models trained on eQASC.

Results and Discussion

Our BERT-based classification models demonstrate a significant improvement in explanation quality, achieving a +14% absolute F1 increase over strong retrieval baselines.

The GRC representation proved particularly effective for model robustness. While standard BERT-based models are highly sensitive to minor wording changes—maintaining consistency only 0.23% of the time under perturbation—the BERT-GRC model achieved a consistency rate of 40.8%. Furthermore, the models showed strong zero-shot generalization to the eOBQA dataset, with AUC-ROC scores ranging from 0.86 to 0.89, suggesting that the learned reasoning patterns are broadly applicable across different scientific domains.

Conclusion

While our models show substantial progress in identifying valid reasoning chains, retrieval remains a significant bottleneck. In many cases, the failure to provide a valid explanation stems from the initial retrieval step's inability to find the necessary facts. Future research will explore extending these methods to longer reasoning chains and improving the precision of entity identification for delexicalization.

Summary updated

Abstract

Despite the rapid progress in multihop question-answering (QA), models still have trouble explaining why an answer is correct, with limited explanation training data available to learn from. To address this, we introduce three explanation datasets in which explanations formed from corpus facts are annotated. Our first dataset, eQASC, contains over 98K explanation annotations for the multihop question answering dataset QASC, and is the first that annotates multiple candidate explanations for each answer. The second dataset eQASC-perturbed is constructed by crowd-sourcing perturbations (while preserving their validity) of a subset of explanations in QASC, to test consistency and generalization of explanation prediction models. The third dataset eOBQA is constructed by adding explanation annotations to the OBQA dataset to test generalization of models trained on eQASC. We show that this data can be used to significantly improve explanation quality (+14% absolute F1 over a strong retrieval baseline) using a BERT-based classifier, but still behind the upper bound, offering a new challenge for future research. We also explore a delexicalized chain representation in which repeated noun phrases are replaced by variables, thus turning them into generalized reasoning chains (for example: "X is a Y" AND "Y has Z" IMPLIES "X has Z"). We find that generalized chains maintain performance while also being more robust to certain perturbations.
BibTeX
@inproceedings{jhamtani2020learning,
  author = {Harsh Jhamtani and Peter Clark},
  title = {{Learning to Explain: Datasets and Models for Identifying Valid Reasoning Chains in Multihop Question-Answering}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2020},
  url = {https://openreview.net/forum?id=SeewS0PzCwr},
}
Figure