F1 is Not Enough! Models and Evaluation Towards User-Centered Explainable Question Answering

Hendrik Schuff, Heike Adel, Ngoc Thang VuEMNLP 2020 · DOI

Abstract

Explainable Question Answering (XQA) systems often suffer from a "faithfulness" gap: they might provide a correct answer but an explanation that has nothing to do with how that answer was reached. This paper introduces a "Select & Forget" architecture and two novel metrics—FARM and LOCA—that ensure answers and explanations are tightly coupled, significantly improving a human user's ability to trust and evaluate the system.

The Problem: The Faithfulness Gap

In the world of Explainable AI (XAI), we want models to not just give us an answer, but to show their work. In Question Answering (QA), this usually means highlighting the specific sentences (supporting facts) used to find the answer.

The authors identify a major flaw in current XQA research: most models treat "answering" and "explaining" as two separate tasks trained in parallel. This leads to two confusing scenarios for users:

  1. Silent Facts: The model uses a piece of information to find the answer but doesn't include it in the explanation.
  2. Unused Facts: The model provides an explanation that looks good but wasn't actually used to generate the predicted answer.

Standard evaluation metrics like $F_1$ scores fail to catch these issues. A model can get a high $F_1$ score by matching the "correct" explanation even if it arrived at the answer through entirely different, unstated reasoning. This mismatch destroys user trust, especially in critical fields like medicine or finance.

Methods: Ensuring Faithfulness by Design

The authors propose two modeling improvements and two new evaluation metrics:

Modeling

  • Select & Forget (S&F) Architecture: Instead of predicting the answer and explanation simultaneously, this hierarchical model first selects relevant facts and then forgets (masks) everything else. The answer prediction head only sees the selected facts, structurally forcing the answer to be derived from the explanation.
  • Answer-Fact Coupling Regularizer: A new loss function term that explicitly penalizes the model when the answer and explanation are uncoupled. It rewards the model for being "right for the right reasons."
The Select & Forget (S&F) architecture diagram showing the flow from fact selection to answer prediction.

Evaluation

  • FARM (Fact-Removal Score): This metric measures how much the model's answer changes when you remove facts from its own explanation. If the answer stays the same after removing the "explanation," the explanation wasn't faithful.
  • LOCA (Answer-Location Score): A simple check to see if the predicted answer string is actually contained within the sentences the model provided as an explanation.

Datasets

The researchers used HotpotQA, a benchmark for multi-hop question answering. It requires models to find and combine information from multiple Wikipedia articles and provides crowd-sourced "supporting facts" to serve as ground-truth explanations.

Key Results and User Study

In a user study with 40 participants, the researchers compared their proposed models against standard baselines.

User study results comparing models across metrics like correct user ratings, false positives, and overestimation.
  • Better User Judgment: The proposed models increased the users' ability to correctly judge whether the system was right or wrong by about 10% compared to the baseline (Figure 6a).
  • Reduced Overestimation: The new models significantly reduced "overestimation," where users mistakenly trust a wrong answer because the explanation looks plausible (Figure 6c). The S&F model even led users to be slightly skeptical, which is safer in high-stakes environments.
  • Metric Correlation: The new metrics (FARM and LOCA) showed a much stronger correlation with actual human behavior and trust than standard $F_1$ scores. While $F_1$ scores for the new models were sometimes slightly lower, the "usefulness" to humans was much higher.

Key Insights

  • $F_1$ is a False Incentive: High $F_1$ scores can mask "unfaithful" models. A model that guesses the right answer but gives a generic, unrelated explanation might score well on $F_1$ but is useless to a human user.
  • Faithfulness > Accuracy for Trust: Users prefer a model that is "honestly wrong." If a model makes a mistake but the explanation clearly shows the faulty logic, users can identify the error. If the model gives a wrong answer with a "correct-looking" but unrelated explanation, users are dangerously misled.

Limitations and Future Work

The "Select & Forget" method faces a "weakest link" problem: if the fact selector misses a crucial fact, the answerer has no way to recover. Future work should explore larger-scale studies across specialized domains like legal or medical contexts to validate if these metrics hold up.

Summary updated

BibTeX
@inproceedings{schuff2020f,
  author = {Hendrik Schuff and Heike Adel and Ngoc Thang Vu},
  title = {{F1 is Not Enough! Models and Evaluation Towards User-Centered Explainable Question Answering}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2020},
  doi = {10.18653/v1/2020.emnlp-main.575},
}
Figure