Relative Importance in Sentence Processing

Nora Hollenstein, Lisa BeinbornACL 2021

TL;DR

This paper investigates how humans and neural language models (like BERT) assign importance to words within a sentence. They found that a model's "saliency" (how much a word contributes to its prediction) aligns better with human reading patterns (measured by eye-tracking) than "attention" mechanisms, suggesting saliency might be a more cognitively plausible way to interpret these models.

What Problem Does This Paper Address?

Understanding how humans process language is a long-standing goal in psycholinguistics. A key aspect of this is determining which words in a sentence are most important for comprehension. Researchers use eye-tracking to measure this by observing how long people fixate on words, assuming longer fixations indicate higher importance.

In parallel, the field of Natural Language Processing (NLP) has developed powerful neural language models. These models also implicitly assign importance to words, often through mechanisms like "attention" or by calculating "saliency" scores. However, it's unclear if these computational measures of importance reflect how humans actually process language. Previous work has shown that attention mechanisms in models don't always align with human reading patterns. This paper aims to bridge this gap by comparing human eye-tracking data with two computational measures of word importance: attention and saliency.

Methods

The researchers employed a data-driven approach to compare human and model importance assignments.

  • Human Importance: They used eye-tracking data, specifically measuring the relative fixation duration on each word. This metric is considered a good proxy for how much processing effort a word requires, abstracting away from individual reading speeds.
  • Model Importance:
    • Attention: They extracted attention weights from the last layer of transformer-based language models. Attention mechanisms are designed to allow models to focus on different parts of the input when processing information.
  • Saliency: They used gradient-based saliency methods. These methods calculate how much a change in an input word's representation affects the model's output (e.g., its prediction of a masked word). This is analogous to identifying which parts of an image are most important for a classification task.

They then calculated the Spearman correlation between these human and model importance measures for individual sentences and averaged across sentences.

Datasets

The study utilized two existing eye-tracking corpora:

  • GECO: Contains eye-tracking data from 14 native English speakers reading the novel "The Mysterious Affair at Styles."
  • ZuCo: Contains eye-tracking data from 30 native English speakers reading full sentences from movie reviews and Wikipedia articles.

The language models used were pre-trained on large English text corpora.

Key Insights and Ideas

  • Saliency over Attention: The most striking finding is that saliency scores from language models correlate significantly better with human reading patterns (fixation durations) than attention weights do. This suggests that saliency might be a more cognitively plausible metric for understanding how these models process language.
  • Linguistic Features Matter: Both human readers and the BERT model (using saliency) assign higher importance to content words (nouns, verbs, adjectives) compared to function words (prepositions, articles). This aligns with linguistic intuition.
  • Word Length and Frequency: Word length and frequency are strong correlates of human fixation duration. Longer words naturally take longer to read, and high-frequency words are often processed more quickly or even skipped. The BERT model's saliency also shows correlations with these features, though the relationship is more complex when analyzed at the token level.
  • Incremental Processing: The paper touches on the idea that humans process language incrementally (left-to-right). While language models can look ahead, the study's findings suggest that saliency might better capture this incremental aspect of human processing than attention.

Key Results

  • Spearman Correlation: Table 1 shows that the Spearman correlation between human relative fixation duration and model saliency was consistently higher across different models (BERT, ALBERT, DistilBERT) and corpora (GECO, ZuCo) compared to the correlation with attention weights. For example, for BERT on the GECO corpus, saliency achieved a correlation of 0.68, while attention was only 0.11.
  • Word Class Importance: Figure 2 visually demonstrates that both humans and BERT saliency assign higher importance to content words over function words.
  • Length and Frequency Correlations: Table 2 shows moderate to strong correlations between word length and frequency with relative importance for both humans and BERT, with saliency showing stronger correlations with frequency.

Limitations and Future Work

  • English-Centric: The study was conducted solely on English data. The authors acknowledge that it's unclear if these findings generalize to other languages and plan to investigate this with non-English eye-tracking data.
  • Token-Level vs. Sentence-Level: While token-level analyses are valuable, the authors note that establishing semantic links across sentences remains a challenge for language models, and human processing is inherently sequential.
  • Saliency Variants: The authors suggest exploring different variants of saliency calculation in future work.
  • Model Architectures: They plan to extend token-level analyses to syntactic phenomena and cross-sentence effects, and investigate how models encode importance for syntactically correct but semantically meaningless sentences. They also propose applying the approach to left-to-right models like GPT-2, potentially using only the first fixation duration for analysis.

Summary updated

BibTeX
@inproceedings{hollenstein2021relative,
  author = {Nora Hollenstein and Lisa Beinborn},
  title = {{Relative Importance in Sentence Processing}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
}
Figure