Do Explanations Help Users Detect Errors in Open-Domain QA? An Evaluation of Spoken vs. Visual Explanations

Ana Valeria Gonzalez, Gagan Bansal, Angela Fan, Yashar Mehdad, Robin Jia, Srinivasan Iyer

Introduction

As Open-Domain Question Answering (ODQA) systems like Alexa and Google Search become ubiquitous, the need for "appropriate reliance"—knowing when to trust or reject an answer—becomes critical. While Explainable AI (XAI) aims to bridge this gap, research often overlooks the practical utility of explanations in real-world tasks and the specific constraints of different modalities. This study investigates whether natural language explanations help users identify system errors and how the effectiveness of these explanations varies between visual and spoken interfaces.

The researchers identify two major gaps in current XAI literature: the Evaluation Gap, where research focuses on proxy tasks rather than end-to-end utility, and the Modality Gap, where visual-centric findings are assumed to apply to voice-based interactions despite the higher cognitive load of listening.

Experimental Design

The researchers conducted a large-scale user study with over 500 Amazon Mechanical Turk participants to evaluate how different explanation types affect a user's ability to detect errors. The study compared several conditions against a baseline of just the answer (BASE) and a baseline including a calibrated confidence score (CONF).

The explanation conditions included:

  • EXT-SENT (Extractive Sentence): A single sentence from the source text containing the answer.
  • EXT-LONG (Extractive Paragraph): A full paragraph of context.
  • ABS (Abstractive): A human-simulated summary of the evidence.

Participants were split into two groups: Visual, where they read the text, and Spoken, where they listened to the information via a Text-to-Speech engine. To ensure the "CONF" baseline was competitive, the authors used temperature scaling to provide honest, calibrated confidence scores.

Interface examples for the user study comparing spoken and visual modalities.

The study utilized the Natural Questions (NQ) corpus, focusing on factoid questions. A state-of-the-art Dense Passage Retrieval (DPR) model was used for evidence retrieval and answer generation. The researchers manually filtered the dataset to remove ambiguous questions, ensuring that user performance was a true measure of explanation quality.

Key Results

The study found that natural language explanations significantly improve error detectability compared to providing only a confidence score. However, the optimal explanation strategy is highly dependent on the modality.

Accuracy of error detectability across different explanation conditions and modalities.
  • Spoken Modality: Short, punchy sentences (EXT-SENT) were most effective, achieving 75.6% accuracy. Longer explanations (EXT-LONG) led to cognitive overload, resulting in lower performance (70.4%).
  • Visual Modality: Detailed paragraphs (EXT-LONG) were the most beneficial, reaching 77.6% accuracy, as users could skim and re-read the text at their own pace.
  • The Confidence Baseline: Calibrated confidence scores (CONF) proved to be a strong baseline (68.1% in spoken), suggesting that explanations must be high-quality to provide additional value over a simple percentage.
  • Abstractive Failure: Surprisingly, summarized explanations (ABS) performed poorly in the spoken modality (71.3%). The high information density of a summary appears harder to process by ear than simpler, more redundant natural sentences.

Discussion and Insights

A primary challenge identified was the "plausibility trap," where users were misled by authoritative-sounding snippets that did not actually support the specific answer. Users often failed to catch subtle mismatches between the evidence and the generated answer if the tone was convincing.

The findings also highlight a discrepancy between subjective preference and objective performance. Users often rated the "answer only" condition as helpful, despite it yielding the lowest actual accuracy. This underscores the importance of measuring actual user accuracy rather than just user preference in XAI research.

Future work should explore Adaptive Explanations, which could tailor the level of detail based on model confidence or user requests, and investigate the use of multiple evidence sources to further improve trust and reliability in real-world settings.

Summary updated

BibTeX
@inproceedings{gonzalez2021do,
  author = {Ana Valeria Gonzalez and Gagan Bansal and Angela Fan and Yashar Mehdad and Robin Jia and Srinivasan Iyer},
  title = {{Do Explanations Help Users Detect Errors in Open-Domain QA? An Evaluation of Spoken vs. Visual Explanations}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.18653/v1/2021.findings-acl.95},
}
Figure