Collecting high-quality adversarial data for machine reading comprehension tasks with humans and models in the loop

Damian Y. Romero Diaz, Magdalena Anioł, John CulnanNAACL 2022

TL;DR

This paper details the authors' experience as annotators in a "Dynamic Adversarial Data Collection" (DADC) task for machine reading comprehension. They found that while DADC is promising for creating robust models, the current F1 metric for evaluating success can lead to false negatives, and the process can be time-consuming and lead to unnatural question generation.

What Problem Does This Paper Address?

Machine reading comprehension (MRC) models have become incredibly powerful, often surpassing human performance on standard datasets. However, they can be brittle and fail spectacularly when presented with "adversarial" inputs – inputs specifically designed to trick them. To combat this, researchers are developing "Dynamic Adversarial Data Collection" (DADC) paradigms where humans and models collaborate in real-time to create challenging datasets. This paper addresses the practical challenges and insights gained from participating in such a DADC task, specifically focusing on the creation of adversarial question-answer pairs for extractive MRC.

Methods

The authors participated in Track 1 of the First Workshop on Dynamic Adversarial Data Collection (DADC). Their approach involved a quasi-experimental design with three annotators (postgraduate students in linguistics and NLP) who created question-answer pairs designed to fool a state-of-the-art MRC model. They analyzed their performance using a single-annotator setup versus a multi-annotator setup. Key analyses included:

  • Quantitative Analysis: Examining the "model fooled" ratio across sessions and annotator groups, and calculating the mean time spent per successful adversarial attack.
  • Qualitative Analysis: Assessing the perceived difficulty of the task based on passage topics and identifying effective adversarial question-creation strategies.
  • Confidence Correlation: Measuring annotator agreement on their confidence in fooling the model.

Datasets

The primary dataset used was generated through the Dynabench platform as part of the DADC shared task. This involved novel annotation passages from various topics. The paper does not introduce a new benchmark dataset but rather discusses the data generated during the task.

Key Insights and Ideas

  • F1 Metric Limitations: The standard F1 word-overlap metric used by Dynabench often resulted in "false negatives," where the model's answer was considered successful despite being incomplete or incorrect. This was particularly problematic when passages contained similar phrases or when the model was biased towards shorter answers.
  • Annotator Fatigue and Efficiency: While multi-annotator sessions were intended to improve quality, they often led to longer annotation times per successful attack. This was attributed to potential disagreements and the tedious nature of the task.
  • Topic Influence: The authors found it easier to create successful adversarial questions for topics they were familiar with (music, science, technology) compared to others (comics, sports, video games). Narrative passages were also more conducive to question creation than data-heavy ones.
  • Effective Strategies: Successful adversarial attacks often involved questions requiring multi-hop reasoning, holistic understanding of the passage, common-sense reasoning, or exploiting nuances in language (e.g., using synonyms or different sentence structures).
  • Unnatural Questions: A significant challenge was generating questions that were both adversarial and natural-sounding, a common issue in DADC that can lead to datasets not reflecting real-world scenarios.

Key Results

  • Annotation Results: Out of 100 total annotation results before verification, 45 were marked as "model fooled" and 43 as "model not fooled" by Dynabench. 10 were false negatives and 2 were false positives.
  • Efficiency: Single-annotator sessions were more efficient, taking an average of 25'37" per successful attack, compared to 46'17" for two-annotator sessions and 1h 13' for three-annotator sessions.
  • Model Fooling Ratio: The model fooling ratio varied by topic, with literature (0.71) and science (1.00) showing higher success rates, while comics (0.33) and sports (0.35) were lower.
  • Confidence Correlation: Spearman rank correlation coefficients between annotators' confidence scores ranged from weak to moderate, suggesting some level of disagreement in their perceived ability to fool the model.

Limitations and Future Work

  • Limited Data: The authors acknowledge that their analysis is based on a limited number of examples, preventing strong conclusions.
  • F1 Metric Improvement: They suggest training a language model to replace or augment the F1 metric for better evaluation of adversarial success.
  • Interface Enhancements: Future DADC interfaces could benefit from features like a "naturality score" for generated questions, visualizations of dataset statistics, and tools for managing annotation strategies.
  • Generative QA: The authors note that applying DADC to generative QA tasks might offer more value, as extractive QA can lead to skewed, unnatural examples.
  • Domain Expertise: Recruiting domain experts for specific sub-domains and using crowdsourced workers for general knowledge texts could improve efficiency.
  • Teamwork: Working in teams for brainstorming and refining questions was found to be more engaging and less tedious.

Summary updated

BibTeX
@inproceedings{diaz2022collecting,
  author = {Damian Y. Romero Diaz and Magdalena Anioł and John Culnan},
  title = {{Collecting high-quality adversarial data for machine reading comprehension tasks with humans and models in the loop}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
}

Loading related content...

Figure