Abstract
Human annotators often employ cognitive heuristics to minimize effort, creating systematic artifacts in datasets. This study investigates the link between annotator psychology and data quality in reading comprehension tasks. By introducing "annotator heuristic traces," we demonstrate that low-effort strategies correlate with psychological profiles and produce data that biased models exploit. These findings highlight a "cascading bias" where human shortcuts become model shortcuts, compromising generalization.
Introduction
NLP models are notorious "shortcut learners," often achieving high performance by exploiting spurious correlations rather than understanding language. While much research focuses on mitigating these artifacts in models, this paper investigates their origin in human behavior. We hypothesize that cognitive heuristics—mental shortcuts used to save time and effort—lead annotators to create predictable data artifacts that models subsequently learn to exploit.
Methodology: Measuring Heuristic Traces
To quantify low-effort strategies, the authors developed "annotator heuristic traces" categorized by psychological principles:
- Satisficing: Measured via "low time" (completion speed) and "low effort" (keystroke-to-length ratio).
- Availability Heuristic: Identified through "first option bias" (correct answer placement) and "serial position" (reliance on specific passage segments).
- Representativeness: Tracked via "word overlap" and "copying" from the source text.
These traces were validated against the Cognitive Reflection Test (CRT), which assesses an individual's tendency to override intuitive, heuristic-driven responses with deliberative reasoning.
Results: The Psychological Link and Model Exploitation
The study found a significant correlation between an annotator's psychological profile and their data-writing behavior. Annotators who scored lower on the CRT were more likely to exhibit high heuristic traces in their work. The first principal component (PCA) of these traces proved to be the strongest predictor of an annotator's psychological profile.
Crucially, these human shortcuts "cascade" into the models. Biased models—such as "no-passage" baselines that should theoretically perform at chance—showed significantly higher precision on questions written by heuristic-heavy annotators. As the data pool moved from high-heuristic to low-heuristic examples, the performance of these biased models dropped sharply, indicating that the models were indeed latching onto human-generated shortcuts.
Impact on Data Quality and Generalization
Heuristic-seeking behavior fundamentally alters data quality. Questions from these annotators are more likely to be unsolvable, invalid, or solvable through simple word-matching rather than complex reasoning. This creates a generalization gap: models trained on heuristic-rich data perform significantly worse when evaluated on cleaner, more robust datasets. For instance, models trained on a "heuristic" split saw a performance drop of several percentage points compared to those trained on random splits.
Conclusion
The findings suggest that dataset artifacts are not random noise but systematic reflections of human cognitive shortcuts. To build more robust AI, researchers should monitor heuristic traces during the annotation process. Identifying these behaviors early allows for better worker training or the creation of more challenging datasets that force models to learn genuine linguistic patterns rather than human-provided "cheats."
Summary updated