TL;DR
Current question answering (QA) systems assume retrieved web documents are truthful, making them fragile when confronted with conflicting information. This paper introduces CONTRAQA, a benchmark of over 10,000 human- and model-generated contradicting contexts, demonstrating that state-of-the-art QA models suffer a 20–30 point drop in Exact Match (EM) when exposed to misinformation. To counter this, the authors propose an integrated defense mechanism combining passage reading with fake-context detection.
What Problem Does This Paper Address?
Standard open-domain QA pipelines retrieve documents from the web and deploy reader models to extract answers. Most benchmarks assume these passages are clean and reliable (often relying on Wikipedia). In the real world, however, the web is rife with misinformation—from honest human factual errors to targeted propaganda and machine-generated hallucinations.
When a QA system retrieves multiple documents that directly contradict each other, which one does it believe? Prior research in QA robustness has primarily examined adversarial perturbations (preserving original meaning) or out-of-domain drift. This work investigates how QA models cope when true contexts are mixed with plausible, contradictory misinformation.
Methods
To evaluate and defend against contradictory contexts, the authors introduce:
- BART-FG (Fake Contexts Generator): A controlled generation model that introduces realistic contradictions through an iterative three-step loop:
- Constituency Parsing: Syntactically parses sentences to extract non-terminal constituent phrases (e.g., noun or prepositional phrases).
- Constituency Masking: Masks key semantic components (dates, locations, entities) rather than random tokens.
- Mask Filling: Uses a BART model pre-trained on Wikipedia via a novel self-supervised objective, Gap Constituency Filling (GCF), which conditions on local and global context to fill the masked constituent with plausible alternative information.
- Misinformation-Aware QA (Defense): To mitigate deception, the authors train a binary discriminator (RoBERTa-large) to score a passage’s trustworthiness ($R_i$). When answering a question, the final confidence score $P_i$ for a candidate span is a linear combination of the reader's extraction confidence ($S_i$) and the trust score: $$P_i = \lambda \cdot S_i + (1 - \lambda) \cdot R_i$$
Datasets
The authors construct CONTRAQA, built on top of SQuAD 1.1:
- Corpus: 10,026 context paragraphs (8,000 training, 2,026 validation).
- Structure: Each real context ($C^R$) is paired with four contradictory contexts ($C^F$): one human-written (collected via Amazon Mechanical Turk with guidelines to alter key facts while maintaining fluency) and three model-generated (via BART-FG).
- Samples: Results in 36,447 training and 10,218 development QA instances where the true context is shuffled alongside misleading distractors.
Key Insights and Ideas
- Contradiction vs. Noise: Models fail not because they struggle with long context or extra text, but because they cannot resolve contradictions. Adding random irrelevant contexts drops performance by only ~9.4%, whereas adding contradictory contexts drops Exact Match by over 41% relatively.
- Humans are Efficient Deceivers: Human annotators achieved higher deception rates with fewer textual edits (19.19% edit distance vs. 24.82%–46.07% for BART-FG). When models made errors, 31.8% of incorrect extractions stemmed from human edits. Humans excel at subtle shifts (e.g., swapping "former" and "latter" or adding exclusionary clauses like "every day but Sunday") that neural models rarely produce.
- Contradiction vs. Detectability Trade-off: Increasing the modification steps ($K$) in BART-FG yields more aggressive contradictions, lowering QA accuracy. However, more edits leave statistical traces that make the fake passage easier for detectors to spot (accuracy rising from 78.04% at $K=1$ to 90.10% at $K=3$).
Key Results
- Performance Collapse: Across BERT-base, RoBERTa-base, RoBERTa-large, and Span-BERT, performance plunged by 23.4 to 25.91 absolute EM points. For RoBERTa-large, EM plummeted from 87.25% on clean SQuAD to 61.34% on Contra-QA.
- Linear Scaling of Confusion: As the number of fake contexts grew from $N=0$ to $N=4$, RoBERTa-large's EM fell almost linearly from 87.25% down to 61.34%.
- Defense Recovery: Incorporating the fake context detector (trained on 10,000 labeled paragraphs) restored RoBERTa-large's EM from 61.34% to 71.92% (a +17.2% relative gain).
Limitations and Future Work
- Detector Data Dependency: The defense relies heavily on large amounts of in-domain labeled data; with only 500 training examples, the performance gain shrinks to +4.0% EM. In open-world settings, obtaining high-quality labeled examples of emerging misinformation is difficult.
- Narrow Misinformation Scope: The benchmark focuses strictly on direct factual contradictions of existing SQuAD texts, leaving out rumors, clickbait, and complex multi-document propaganda campaigns.
- Scope of QA Evaluation: Experiments evaluate extractive readers under closed candidate pools rather than a full, end-to-end open-domain retrieval pipeline operating over the live web.
Summary updated