An Analysis of Frequency- and Memory-Based Processing Costs

Marten van Schijndel, William SchulerNAACL 2013

TL;DR

This paper investigates whether memory limitations, rather than just word frequency, explain why certain grammatical structures are less common and harder to process. By analyzing eye-tracking data with a novel hierarchical sequential model, they found that memory-based measures significantly predict reading times, even after accounting for frequency.

What Problem Does This Paper Address?

Language processing is known to be influenced by word and construction frequency. However, the reason for this frequency variation has been less clear. One hypothesis, the "grounding hypothesis," suggests that languages evolve to minimize processing costs, implying that cognitive limitations like working memory might be the underlying cause of certain structures being less frequent. This paper aims to disentangle the effects of frequency and memory limitations on language processing, specifically exploring how working memory constraints might influence reading times and, by extension, the evolution of language.

Methods

The authors employ a hierarchic sequential prediction model that implements a left-corner parser. This model represents sentence parsing as a series of connected components of phrase structure trees, managed by hidden random variables. The core innovation lies in defining parsing operations (like initiating or terminating a new embedded structure) probabilistically. This allows them to directly compute measures related to dependency locality (DLT), ACT-R's confusability, hierarchic sequential prediction, and embedding difference. These measures are then used to predict reading times from eye-tracking data.

Datasets

The primary dataset used for evaluation is the Dundee eye-tracking corpus. This corpus contains 2388 sentences of naturally occurring British English news text, with eye-tracking data from 10 native English speakers, resulting in approximately 151,331 subject-duration pairs after filtering.

Key Insights and Ideas

  • Memory as a Driver of Frequency: The study provides evidence that memory limitations might be a significant factor in why certain grammatical constructions are less frequent and thus more surprising.
  • Disentangling Frequency and Memory: The research successfully isolates memory effects from frequency effects, demonstrating that memory load remains a significant predictor of reading time even when frequency measures are controlled for.
  • Initiating Embeddings Slows Processing: The initiation of a new embedded structure (F+L- operation in their model) was found to be a strong predictor of increased reading time, supporting theories that suggest encoding or storage costs associated with managing nested dependencies.
  • Completing Embeddings Facilitates Processing: Conversely, completing an embedded structure (F-L+ operation) was associated with reduced reading time, suggesting a facilitation effect. This contradicts some older theories of dependency locality but aligns with hypotheses involving dynamic recruitment of processing resources or a reduction in memory load.
  • Sequential Cueing is Less Predictive: Simple sequential cueing operations (F-L- and F+L+) were not significant predictors of reading times when frequency effects were accounted for, suggesting that these learned associations are more akin to frequency effects than memory-based costs.

Key Results

The study found that the F+L- (initiating a new embedded component) operation was a strong positive predictor of reading time durations. The F-L+ (integrating a subordinate component) operation was also a significant predictor, but with a negative coefficient, indicating a facilitation effect. These findings were robust even after residualizing for baseline frequency measures like surprisal and entropy reduction. The spillover versions of these measures were not significant predictors.

Limitations and Future Work

The authors acknowledge that the negative integration cost finding (F-L+) could be evidence of anti-locality or processing momentum from dynamic resource recruitment. They suggest future work could investigate this by examining whether the facilitation is driven by longer embeddings or a general effect when completing embeddings. They also note the discrepancy between their findings and previous studies that found positive integration costs, suggesting that factors like the use of naturalistic stimuli versus constructed ones, or the presence of ambiguities, might play a role. The paper also points to the need for more precise frequency metrics or further investigation into the distance between encoding and integration actions to fully understand anti-locality effects.

Summary updated

BibTeX
@inproceedings{schijndel2013analysis,
  author = {Marten van Schijndel and William Schuler},
  title = {{An Analysis of Frequency- and Memory-Based Processing Costs}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2013},
}
Figure