Human Attention Maps for Text Classification: Do Humans and Neural Networks Focus on the Same Words?

Cansu Sen, Thomas Hartvigsen, Biao Yin, Xiangnan Kong, Elke A. Rundensteiner

Introduction

In Natural Language Processing (NLP), "attention" mechanisms are frequently used to provide interpretability, suggesting which parts of an input text a model prioritizes for its predictions. However, a significant debate persists: does this mathematical weighting truly reflect human-like reasoning, or is it merely a computational artifact? This research addresses this gap by introducing YELP-HAT, a large-scale dataset of human attention maps, to quantify the alignment between human cognitive focus and neural network attention.

The YELP-HAT Dataset

To establish a ground truth for human attention, the researchers developed the YELP-HAT (Human ATtention) dataset. Using Amazon Mechanical Turk, participants were presented with restaurant reviews and asked to highlight specific words that justified their sentiment classification (positive or negative).

The crowdsourcing interface used to collect human attention maps, where users highlight words reflecting the review's sentiment.

The dataset consists of 15,000 human attention maps derived from 5,000 unique Yelp reviews. To account for subjectivity, three independent human maps were collected for each review. The data is categorized by review length—50, 100, and 200 words—allowing researchers to observe how increased text complexity affects both human and machine focus.

Methodology

The study compared standard LSTM models (uni-directional and bi-directional) with soft attention against "Rationale" models, which use hard attention to select discrete text snippets. To measure similarity, the authors proposed several metrics:

  • Behavioral Similarity: Measured via Area Under the Curve (AUC) to compare the overlap between human-selected words and machine-weighted words.
  • Lexical Similarity: Evaluated whether humans and machines prioritize the same grammatical categories, such as adjectives or nouns.
  • Context-dependency (CSSR): Analyzed how often models mistakenly focus on sentiment-bearing words that are negated by their context (e.g., focusing on "good" in "not good").

Key Findings and Results

The research yielded several critical insights into the nature of machine attention:

1. The Necessity of Bi-directionality

Bi-directional RNNs demonstrated a significantly higher alignment with human focus than uni-directional models. On shorter reviews (Yelp-50), bi-directional models achieved a behavioral similarity score (AUC) of 0.79, whereas uni-directional models scored only 0.59. This suggests that the ability to process text in both directions mimics human behaviors like re-reading and looking ahead for context.

2. The Consensus Effect

Machine attention aligns most closely with the Consensus Attention Map (CAM)—the set of words where all three human annotators agreed. This indicates that neural networks are effective at identifying objective sentiment markers but struggle to capture the subjective nuances found in individual human maps.

3. The Length Penalty

As text length increases, the similarity between human and machine attention degrades. In longer reviews, both humans and models encounter more "noise," leading to divergent focus areas.

Quantitative comparison of similarity metrics across different review lengths and model architectures.

4. Accuracy vs. Interpretability

While humans maintain a lead in classification accuracy (~94-96%), bi-directional models are competitive (~91-93%). However, "Rationale" models, despite being designed for readability, saw a sharp decline in both accuracy and human-similarity as text length increased, highlighting a trade-off between hard-coded interpretability and performance.

Conclusion and Future Directions

This study confirms that while modern neural networks—particularly bi-directional ones—can mirror human lexical patterns, they are still susceptible to "distraction" in complex texts. The YELP-HAT dataset provides a foundation for supervised attention, where models could be explicitly trained to "look" at text through a human lens. Future research will determine if these findings extend beyond sentiment analysis to more complex reasoning tasks like natural language inference.

Summary updated

BibTeX
@inproceedings{sen2020human,
  author = {Cansu Sen and Thomas Hartvigsen and Biao Yin and Xiangnan Kong and Elke A. Rundensteiner},
  title = {{Human Attention Maps for Text Classification: Do Humans and Neural Networks Focus on the Same Words?}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-main.419},
}
Figure