Understanding Attention in Machine Reading Comprehension

Yiming Cui, Wei-Nan Zhang, Wanxiang Che, Ting Liu, Zhigang ChenarXiv 2021 · OpenReview

Abstract: Transformer-based models like BERT have achieved human-level performance on Machine Reading Comprehension (MRC) tasks, yet their internal decision-making processes remain opaque. This study investigates these mechanisms by decomposing the attention matrix into four functional zones: Question Understanding ($Q^2$), Clue Finding ($Q2P$), Answer Verification ($P2Q$), and Passage Understanding ($P^2$). Through systematic masking experiments across English (SQuAD) and Chinese (CMRC 2018) datasets, the authors demonstrate that the model's ability to verify answers ($P2Q$) and understand the passage ($P^2$) are the primary drivers of performance, while the traditionally emphasized "clue finding" ($Q2P$) is largely redundant.

Introduction

Machine Reading Comprehension (MRC) requires a model to read a passage and extract an answer based on a specific question. While Pre-trained Language Models (PLMs) have mastered benchmarks like SQuAD, their internal decision-making process is often viewed as a "black box." This research addresses the ongoing debate in the NLP community regarding whether attention maps provide genuine explanations for model behavior. By analyzing attention through a granular, task-specific lens, the authors seek to identify which parts of the attention mechanism are truly essential for reasoning.

Methodology: The Four Attention Zones

The authors introduce a novel framework for analyzing the $L \times L$ attention matrix by dividing it into four Attention Zones based on the source and target of the attention weights:

  1. $Q^2$ (Question Understanding): The question tokens attending to themselves.
  2. Q2P (Clue Finding): The question tokens attending to the passage.
  3. P2Q (Answer Verification): The passage tokens attending to the question.
  4. $P^2$ (Passage Understanding): The passage tokens attending to themselves.
The four functional attention zones and their conceptual roles in the MRC process.

To test the importance of these zones, the researchers conducted masking experiments, systematically disabling specific zones or tokens (such as [CLS] or [SEP]) during training and inference to measure the resulting performance drop. They also calculated the Pearson correlation between attention scores and final accuracy to determine if high attention weights correlate with task importance.

Key Results: The Criticality of Answer Verification

The most significant finding is the "From Passage" priority. The critical drivers of performance are the $P2Q$ (Answer Verification) and $P^2$ (Passage Understanding) zones.

Performance drop across layers when masking different attention zones for SQuAD and CMRC 2018 datasets. Darker blue indicates a more significant performance collapse.

In SQuAD experiments, removing the $P2Q$ zone caused the Exact Match (EM) score to plummet from 80.6 to 12.7. Conversely, removing the $Q2P$ zone—traditionally thought to be the primary way models find answers—only dropped the score to 79.9, suggesting it is largely redundant. Furthermore, $P2Q$ showed the highest correlation with system performance (0.76–0.77), proving that this specific type of attention is a reliable proxy for model effort.

Insights into Model Behavior

  • Model Redundancy: Large-scale models (like BERT-large or ELECTRA-large) are incredibly robust. Entire attention zones can be disabled in several layers with minimal impact, indicating that larger models store redundant knowledge across layers as a "backup" system.
  • Harder Questions Require Deeper Questioning: When faced with complex reasoning tasks or "Why" questions, models show significantly higher activity in the $Q^2$ zone. This mirrors human behavior: if a question is difficult, the model spends more time re-reading and parsing the question itself.
  • Span-Level Reasoning: Unlike simpler models that just look for start and end pointers, BERT-based models attend to the entire answer phrase, which explains their superior effectiveness in span-extraction tasks.

Limitations and Future Work

While the study covers English and Chinese, the authors acknowledge that morphologically rich languages (like Arabic or Finnish) might exhibit different patterns. Additionally, the findings focus on encoder-based models; future research is needed to see if these insights translate to newer decoder-only models like GPT-4. The authors suggest that these findings could be used to automatically prune unimportant attention heads, creating faster, more efficient, and more explainable models.

Summary updated

BibTeX
@misc{cui2021understanding,
  author = {Yiming Cui and Wei-Nan Zhang and Wanxiang Che and Ting Liu and Zhigang Chen},
  title = {{Understanding Attention in Machine Reading Comprehension}},
  year = {2021},
  url = {https://openreview.net/forum?id=U2iyxv1pcf},
}
Figure