Using MoTR to Probe Agreement Processing in Russian

Metehan Oğuz, Cui Ding, Ethan Wilcox, Zuzanna FuchsUniversität 2025 · DOI

TL;DR

This paper introduces Mouse Tracking for Reading (MoTR) as a novel, cost-effective tool for studying real-time language processing, demonstrating its ability to detect subtle linguistic effects in Russian, a morphologically rich language. The study provides moderate evidence that agreement errors are processed differently depending on whether the agreement is internal (e.g., with adjectives) or external (e.g., with verbs) to the noun phrase, suggesting this distinction is relevant for how we process language.

What Problem Does This Paper Address?

Understanding how humans process grammatical agreement in real-time is crucial for theories of language comprehension. While it's known that agreement errors (e.g., a singular noun with a plural verb) cause processing difficulty, it's unclear if this difficulty varies based on the type of agreement. Specifically, linguists distinguish between internal agreement (within a noun phrase, like an adjective agreeing with a noun) and external agreement (between a noun phrase and a predicate, like a verb agreeing with a noun). This distinction has theoretical implications for how agreement is represented and processed. Previous studies have yielded mixed results, partly due to methodological limitations. This paper aims to address this by:

  1. Investigating whether agreement errors are processed differently based on their internal vs. external status.
  2. Validating Mouse Tracking for Reading (MoTR) as a reliable and cost-effective tool for psycholinguistic research, particularly for languages with complex morphology and non-Latin scripts, and for studying agreement processing.

Methods

The core of this study is the application of Mouse Tracking for Reading (MoTR), a relatively new web-deployable method. In MoTR, participants read text on a screen that is initially blurred. They must move their mouse cursor over the text to reveal it. The system records the cursor's x/y coordinates at a high sampling rate (20 Hz). This movement data is then used to derive incremental reading times (similar to fixation durations in eye-tracking) and other reading measures.

Key aspects of MoTR highlighted:

  • Web-deployable and cost-effective: Allows for larger sample sizes and lower costs compared to lab-based methods.
  • Naturalistic reading: Participants move their mouse to reveal text, mimicking natural reading behavior.
  • Flexibility: Allows for skipping words and regressions, unlike self-paced reading.
  • Data analysis: Uses Bayesian hierarchical models to analyze reading measures, accounting for participant and item variability.

The study also reanalyzes existing eye-tracking data from a previous study (Fuchs et al., 2025) using the same analytical framework for direct comparison.

Datasets

The study used a dataset of Russian sentences specifically designed to test agreement processing. These stimuli were adapted from a previous study (Fuchs et al., 2025) and manipulated to create grammatical and ungrammatical conditions for three types of agreeing elements:

  • Modifying adjectives (internal agreement)
  • Verbs (external agreement)
  • Predicative adjectives (external agreement)

The stimuli focused on inanimate feminine and masculine nouns. Participants also answered comprehension questions to ensure attentive reading.

Key Insights and Ideas

  • MoTR is a viable tool for psycholinguistics: The study successfully demonstrates that MoTR can detect subtle processing differences in agreement errors, even in a morphologically rich language like Russian with a non-Latin script. This opens doors for more accessible and scalable psycholinguistic research.
  • Agreement type matters: There's moderate evidence that processing agreement errors is modulated by whether the agreement is internal or external. Specifically, errors in internal agreement (modifying adjectives) might incur a larger processing cost than errors in external agreement (verbs, predicative adjectives).
  • MoTR shows larger effect sizes than eye-tracking: In a direct comparison, MoTR data yielded larger effect sizes for agreement effects than the reanalyzed eye-tracking data. This suggests MoTR might be more sensitive, potentially due to the manual control required (hand/mouse movements vs. just eye movements).
  • Lexical category is less influential than agreement type: The study found no strong evidence that the processing of agreement errors is modulated by the lexical category (adjective vs. verb) of the agreeing element, suggesting that the type of agreement relationship is a more significant factor.

Key Results

  • Grammaticality effects: Both MoTR and eye-tracking data showed clear evidence that agreement mismatches lead to increased processing effort, particularly in later reading measures (e.g., total duration, regressions).
  • MoTR: Mismatches led to an additional 32 ms in gaze duration and 80 ms in total duration.
  • Eye-tracking: Mismatches led to 13 ms more in gaze duration and 66 ms in total duration.
  • Agreement Type (AgrType) effects:
    • MoTR: Showed a larger negative main effect for AgrType, indicating external agreement (verbs, predicative adjectives) leads to longer reading times and more regressions compared to internal agreement (modifying adjectives).
  • Eye-tracking: Showed a negative effect, but with wider credible intervals.
  • Grammaticality x Agreement Type Interaction (GramxAgrType):
    • MoTR data showed a trend where the mismatch penalty was larger for internal agreement than external agreement, with larger effect sizes than eye-tracking.
  • Eye-tracking data showed a similar trend but with very wide credible intervals that included zero, making the conclusion less certain.
  • A post-hoc analysis on MoTR data specifically for adjectives found a significant mismatch cost for internal agreement (modifying adjectives) but not for external agreement (predicative adjectives).
  • Grammaticality x Lexical Category Interaction (GramxLexCat): No significant interaction was found in either MoTR or eye-tracking data, suggesting lexical category does not modulate agreement error processing.
  • Power Analysis: MoTR demonstrated higher statistical power than eye-tracking for detecting the GramxAgrType interaction, especially for go-past time, suggesting greater sensitivity. Both methods showed low power for detecting the GramxLexCat interaction.

Limitations and Future Work

  • Stimuli limitations: The study used only inanimate feminine and masculine nouns, and a subset of stimuli from a previous study. The exclusion of neuter nouns and animate nouns might limit the generalizability of the findings.
  • Language-specific confounds: While Russian is ideal for this research, there are potential confounds like length differences between modifying and predicative adjectives, and the ambiguity of long-form adjectives in general Russian usage (though constrained in the experiment).
  • Small number of stimuli: The study used a relatively small number of stimuli (8 per condition), which might affect the generalizability of the results.
  • Ancillary noun control: Features (number and gender) in non-critical positions were not systematically controlled, which could introduce minor confounds like proactive interference.

Future Work: The authors suggest further experiments are needed to definitively answer the research questions, particularly to explore the generalizability of findings to animate nouns and neuter genders. They also propose investigating other languages with similar properties but different word orders (e.g., Hebrew) and exploring the potential of MoTR for other linguistic phenomena and languages. The observed differences in effect sizes and power between MoTR and eye-tracking also warrant further investigation.

Summary updated

Abstract

One important distinction in the syntax literature is between agreement that is external to the nominal phrase and agreement that is internal to it (sometimes called concord). How this type of agreement impacts sentence processing, however, is not well understood. In this paper, we ask whether agreement errors are processed differently based on their internal vs. external status. We investigate this question in Russian, a Slavic language that has a rich morphological agreement system and flexible word order, allowing us to control for several confounds. Our results are not fully conclusive but do provide moderate evidence that processing of agreement is modulated by internal vs. external status. We measure real-time language processing using Mouse Tracking for Reading (MoTR), a new web-deployable measurement tool that has been argued to improve over previous methods (e.g., self-paced reading) but has so far been tested only in English, and never for agreement processing phenomena. We find that MoTR can successfully pick up differences in our factorized psycholinguistic experiment in Russian, validating MoTR as a reliable tool for investigating agreement (error) processing. A direct comparison with existing data collected using in-lab eye-tracking-while-reading with similar experimental materials (Fuchs et al., 2025) suggests MoTR data yields larger effect sizes than does eye-tracking data.
BibTeX
@inproceedings{oguz2025using,
  author = {Metehan Oğuz and Cui Ding and Ethan Wilcox and Zuzanna Fuchs},
  title = {{Using MoTR to Probe Agreement Processing in Russian}},
  booktitle = {Universität},
  year = {2025},
  doi = {10.5167/uzh-281794},
}
Figure