Finding Generalizable Evidence by Learning to Convince Q&A Models

Ethan Perez, Siddharth Karamcheti, Rob Fergus, Jason Weston, Douwe Kiela, Kyunghyun Cho

Abstract

Ethan Perez, Siddharth Karamcheti, Rob Fergus, Jason Weston, Douwe Kiela, Kyunghyun Cho. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP).

2019.

Introduction

In passage-based Question Answering (QA), models often struggle to identify the most relevant information within large volumes of text. While performance is improving, the underlying reasoning remains opaque. Furthermore, models frequently fail when encountering "distribution shift"—for instance, when a model trained on short stories is tested on complex high school exam passages.

Researchers address these challenges by developing "evidence agents" that learn to extract the most persuasive sentences from a text to support a specific answer. This study explores whether evidence that convinces a machine learning model is also effective for humans and other models, potentially leading to more robust and interpretable QA systems.

The Judge-Agent Framework

The authors propose a framework consisting of two main components:

  1. The Judge: A pretrained QA model (e.g., BERT) that outputs probabilities for each possible answer choice given a text snippet.
  2. The Evidence Agents: For every possible answer choice, an assigned agent selects a subset of sentences from the original passage to maximize the Judge's probability of selecting its assigned answer.
The Judge-Agent framework: Evidence agents select sentences to convince a Judge model of a specific answer choice.

Two types of agents were evaluated: Search Agents, which use exhaustive search to find the most influential sentences, and Learned Agents, which are BERT-based models trained to predict which sentences will be most convincing. Interaction protocols included Individual selection, Free-for-All competition, and Round Robin pairwise matches.

Experimental Setup

The framework was tested on two challenging multiple-choice datasets:

  • RACE: English reading comprehension exams for middle and high school students, requiring complex reasoning.
  • DREAM: A dialogue-based dataset featuring informal conversations.

These datasets are particularly useful because their "distractor" answers are manually authored to be plausible, forcing agents to find genuinely strong evidence.

Key Insights

  • Convincingness Generalizes: Evidence selected to convince a BERT model also convinced human evaluators. This suggests that "convincingness" is a general property rather than a model-specific quirk.
  • Task Denoising: By forcing models to work with only ~20% of the original text, the agents effectively "denoise" the task, highlighting the core information.
  • Robustness through Selection: Relying on selected evidence improves generalization. A model trained on short passages performed significantly better on long passages when using agent-selected evidence instead of the full, noisy text.
  • Learned vs. Search Agents: Learned agents were more effective at finding "human-valid" evidence, whereas search agents occasionally exploited "adversarial" quirks in the Judge model.

Results

The study found that evidence selected for models is highly persuasive to humans. On the RACE dataset, humans selected the agent-supported answer 42% of the time when shown only one selected sentence (vs. 25% random baseline). For correct answers, the selected evidence helped humans identify the right choice 74.6% of the time.

Human evaluation results: Evidence selected to convince models also significantly influences human choice.

Key performance metrics include:

  • Generalization to Long Passages: BERT accuracy on long RACE passages improved from 44.1% to 50.2% using agent-selected evidence.
  • Harder Questions: Performance on high-school questions improved from 50.7% to 53.0% when using evidence agents.
  • Human Efficiency: Humans maintained ~90% of their full-passage accuracy while reading only 20% of the text, provided it was selected by agents.

Conclusion and Future Work

While the framework shows promise for creating more robust and interpretable QA, it is currently limited to multiple-choice formats. Future research is needed to adapt this to generative QA and to mitigate the risk of "deceptive" agents that might successfully argue for false answers. Ensuring agents remain "honest" while being persuasive is a key challenge for future AI safety research.

Summary updated

BibTeX
@inproceedings{perez2019finding,
  author = {Ethan Perez and Siddharth Karamcheti and Rob Fergus and Jason Weston and Douwe Kiela and Kyunghyun Cho},
  title = {{Finding Generalizable Evidence by Learning to Convince Q\&A Models}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2019},
  url = {https://openreview.net/forum?id=Lp3Y-Haix_B},
}

Loading related content...

Figure