Computationally Rational Saccadic Control: An Explanation of Spillover Effects Based on Sampling from Noisy Perception and Memory

Michael Shvartsman, Richard L. Lewis, Satinder P. SinghCMCL @ ACL 2014 · OpenReview

TL;DR

This paper proposes a computational model where eye movements during reading are controlled by a rational agent balancing immediate perception with memory recall. This model explains "spillover effects" (where processing of one word influences fixation on the next) as an emergent property of this trade-off, even in the absence of parafoveal preview.

What Problem Does This Paper Address?

Reading involves a complex interplay between where our eyes are looking (perception) and what we've just seen (memory). A phenomenon called "spillover effects" has been observed where the time spent looking at a word (fixation duration) is influenced by the properties of the previous word. This suggests that processing doesn't simply stop when the eyes move on. Traditional explanations often relied on "parafoveal preview" – the idea that we're already processing the next word while looking at the current one. However, spillover effects persist even when this preview is blocked. This paper addresses the question of how spillover effects can arise if not solely from parafoveal preview, proposing that it's a consequence of a rational control strategy that strategically uses memory of past input.

Methods

The authors extend an existing adaptive eye-movement control model by incorporating a "noisy memory" buffer. This model uses a "sequential sampling" approach, where decisions (like planning a saccade or shifting attention) are made when a certain evidence threshold is reached. The novelty lies in allowing this sampling process to draw from both noisy perceptual input and noisy memory of past input. The model's "policy" (how it allocates resources between perception and memory) is determined by decision thresholds. They then perform a "computational rationality analysis" by evaluating different policy parameters against a task payoff, essentially asking what strategies are optimal given the model's architecture and the task's demands.

Datasets

The primary dataset used for evaluation is from the List Lexical Decision Task (LLDT), an experiment described in the paper. This task involves participants determining if a list of strings contains only words or a single nonword. The paper also references existing literature and data from other studies on eye movements in reading. No new benchmark datasets are introduced.

Key Insights and Ideas

  • Spillover as a Rational Trade-off: The core idea is that spillover isn't an error, but an emergent property of a rational agent balancing the cost of processing new information against the benefit of recalling past information, especially when that information is noisy.
  • Memory's Role: The model demonstrates that even a simple, noisy memory of recently processed words can lead to spillover effects, suggesting that continued processing of past input is a significant factor.
  • Optimal Policies Can Exhibit Spillover: Counter-intuitively, the best-performing policies (those that maximize task payoff) in the model often exhibit spillover. This suggests that spillover is not necessarily a sign of suboptimal performance.
  • Spillover Policies Don't Perform Worse: When compared to policies that are forced to prioritize perception over memory (thus eliminating spillover), the spillover-capable policies perform equally well or better across various noise settings and speed-accuracy trade-offs.
  • The "Why": Sequential Thresholded Samplers: The paper traces the origin of spillover to the dynamics of sequenced thresholded samplers. When an incorrect hypothesis is initially favored, the system needs to "recover," which takes time and is more likely for low-frequency words, thus delaying subsequent processing.

Key Results

  • The model successfully replicates frequency spillover effects observed in human reading behavior, even in a masked condition that denies parafoveal preview.
  • The top-performing 5% of policies in the model, evaluated by task payoff, showed spillover effects that were consistent with human data.
  • Spillover effects in the model were found to be more pronounced under higher "update noise" conditions, suggesting a rational adaptation to uncertainty.
  • Spillover-capable policies consistently performed as well as or better than spillover-incapable policies across a range of noise settings and speed-accuracy trade-offs.

Limitations and Future Work

  • Minimal Cognitive System: The model is built on a minimal cognitive system capable of performing the LLDT. Future work could explore how these principles apply to more complex reading tasks and higher-level sentence processing.
  • Parafoveal Preview Interaction: While the paper focuses on memory, the role of parafoveal preview in spillover effects could be further explored and integrated into the model.
  • Alternative Architectures: The model uses a serial queuing realization of limited resources. Investigating parallel attention machines that might exhibit similar trade-offs is a potential avenue.
  • Oculomotor Control Integration: The model's saccade planning is directly triggered by perceptual evidence. Integrating it with more autonomous saccade generation models (like SWIFT) is a future direction.
  • Noise Parameter Exploration: While the paper explores a range of noise settings, the highest noise levels (near chance performance) were not fully analyzed, suggesting room for further investigation.

Summary updated

BibTeX
@inproceedings{shvartsman2014computationally,
  author = {Michael Shvartsman and Richard L. Lewis and Satinder P. Singh},
  title = {{Computationally Rational Saccadic Control: An Explanation of Spillover Effects Based on Sampling from Noisy Perception and Memory}},
  booktitle = {CMCL @ ACL},
  year = {2014},
  url = {https://openreview.net/forum?id=SlVU-VJaeFP},
}

Loading related content...

Figure