Towards a Data-Driven Model of Eye Movement Control in Reading

Mattias Nilsson, Joakim NivreCMCL @ ACL 2010 · OpenReview

Abstract

Mattias Nilsson, Joakim Nivre. Proceedings of the 2010 Workshop on Cognitive Modeling and Computational Linguistics. 2010.

Researchers from Uppsala University have developed a machine-learning-based model that predicts both where and when a reader’s eyes move during reading. By combining a transition-based classifier with a temporal model of lexical processing, the system successfully replicates human reading behaviors—such as skipping common words and lingering on difficult ones—on previously unseen text.

Introduction

In psycholinguistics, understanding eye movement control is essential for uncovering how the brain processes language. Traditional computational models, such as E-Z Reader or SWIFT, are typically "theory-first," built on specific psychophysiological assumptions with parameters fitted to entire datasets. However, these models often lack rigorous generalization testing on unseen data. Furthermore, previous data-driven efforts often focused solely on the spatial aspect ("where") of eye movements. This research addresses these gaps by integrating both the "when" and "where" into a single, data-driven framework using machine learning principles.

Methodology

The authors propose a model defined by four concurrent or sequential processes: lexical processing ($L$), saccade initiation delay ($D$), motor programming ($M$), and saccade execution ($S$). Lexical processing time is modeled as a linear function of a word's length and its log-frequency. The "where" decision is treated as a classification task using logistic regression, while the "when" decision is triggered by the completion of lexical processing.

Model parameters for lexical processing and eye movement timing.

The model was trained and evaluated using the English section of the Dundee Corpus, which contains eye-tracking records of subjects reading editorials. A strict separation between training (1,911 sentences), development (237 sentences), and test (231 sentences) sets was maintained to ensure the model's ability to generalize to new text.

Key Results

The authors evaluated three versions of the model ($M_1$ through $M_3$). $M_3$, which utilized both word length and frequency, was the most effective. The target selection classifier achieved 72% accuracy in predicting the next word a reader would fixate on.

Observed vs. predicted gaze durations and skipping probabilities across frequency classes.

As shown in the results, the model successfully replicated the "frequency effect," where readers spend less time on common words. It also accurately predicted skipping probabilities, showing that high-frequency words like "the" are skipped much more often than rare words. The model achieved a Root Mean Square Error (RMSE) of 0.26 on the held-out test set, demonstrating a strong fit for behavioral modeling.

Discussion and Limitations

A major takeaway is that the decision of where to move can be modeled as an automated routine learned through practice, while timing is dynamically influenced by immediate cognitive difficulty. Despite its success, the model is currently a baseline with several limitations: it does not account for regressions (moving backward), within-word fixation locations, or higher-level linguistic features like syntax and semantics. Future work will focus on incorporating more sophisticated natural language processing (NLP) features to account for these complexities.

Summary updated

BibTeX
@inproceedings{nilsson2010towards,
  author = {Mattias Nilsson and Joakim Nivre},
  title = {{Towards a Data-Driven Model of Eye Movement Control in Reading}},
  booktitle = {CMCL @ ACL},
  year = {2010},
  url = {https://openreview.net/forum?id=NKRRuTwBKrm},
}
Figure