From eye movements to scanpath networks: A method for studying individual differences in expository text reading

Xiaochuan Ma, Yikang Liu, Roy B. Clariana, Chanyuan Gu, Ping LiBehavior 2022 · DOI

TL;DR

This paper introduces a novel method for analyzing reading comprehension by transforming eye-tracking data into "scanpath networks." These networks, analyzed using network science metrics, reveal distinct patterns between skilled and less-skilled readers, offering a more nuanced understanding of individual differences in reading than traditional methods.

What Problem Does This Paper Address?

Reading comprehension is crucial for knowledge acquisition, but traditional assessments (like post-reading questions) often provide a single, insufficient score. While eye movements are known indicators of attention and comprehension, analyzing the complex patterns of how readers scan text has been challenging. Existing methods often focus on isolated eye-movement features (like fixation duration) or treat text as a monolithic entity. This paper aims to bridge this gap by using network science to model the dynamic, sequential nature of eye movements during reading, specifically in expository scientific texts, to better understand individual differences in comprehension.

Methods

The core innovation here is the transformation of raw eye-tracking data into scanpath networks. Here's how it works:

  1. Scanpath Generation: Participants' eye movements while reading expository science texts were recorded. Each word fixated was treated as a "node" in a network, and the transitions (saccades) between fixated words became "edges." Reoccurring words were merged into a single node. This creates a directed graph representing the reader's unique journey through the text.
  2. Network Metrics Analysis: A suite of network science metrics were applied to these scanpath networks to quantify their structural properties:
  3. Density: Measures how interconnected the network is (high density suggests fewer skips).
  4. Centrality: Identifies key nodes (words) that act as bridges in the network.
  5. Transitivity: Reflects the tendency for nodes to form triangles, potentially indicating regressive fixations.
  6. Global Efficiency: Measures how efficiently information can travel through the network.
  7. Small-worldness: Assesses a network's balance between local clustering and global integration.

These metrics were then correlated with participants' reading comprehension scores and compared between groups of skilled and less-skilled readers.

Datasets

The study utilized a dataset of eye-tracking data from 52 participants reading five expository science texts. These texts covered topics such as Mars exploration, electrical circuits, permutations/combinations, supertankers, and GPS. The data was collected as part of the "Reading Brain Project L1 Adults."

Key Insights and Ideas

  • Scanpath Networks as a Rich Representation: The study demonstrates that scanpath networks can capture complex information processing strategies during reading, going beyond simple metrics like fixation duration.
  • Network Structure Reflects Comprehension Skill: Distinct network structures emerged for skilled versus less-skilled readers, suggesting that the way a reader navigates text is intrinsically linked to their comprehension ability.
  • Efficiency in Skilled Reading: Skilled readers' networks showed patterns indicative of more efficient processing, such as higher centrality and global efficiency, and lower density and transitivity. This aligns with previous findings that skilled readers skip more words and make fewer regressions.
  • Text Structure Influence: The authors acknowledge that text structure (e.g., sentence-by-sentence presentation) can influence network metrics, but argue that this constraint can also reveal interesting differences in how readers handle information.

Key Results

  • Correlation with Comprehension: Network metrics showed significant correlations with a combined "comprehension ability score." Specifically, centrality was positively correlated, while density, global efficiency, and transitivity were negatively correlated.
  • Group Differences: Skilled readers (top 26) and less-skilled readers (bottom 26) showed significant differences across most network metrics:
  • Skilled readers had higher centrality and lower density, transitivity, and global efficiency.
  • The difference in small-worldness was also significant, with skilled readers exhibiting lower values.
  • Visualizations: Averaged scanpath networks for skilled and less-skilled readers (shown in Figure 5) visually highlighted these differences, with larger nodes and thicker edges representing higher values. Less-skilled readers' networks appeared denser, suggesting more repetitive fixations, particularly on function words.

Limitations and Future Work

  • Sentence-by-Sentence Presentation: The authors acknowledge that presenting text sentence-by-sentence, rather than whole paragraphs, might constrain the natural reading experience and limit the ability to study cross-sentence regressions. However, they suggest this constraint might also reveal interesting patterns related to working memory.
  • Future Directions: The authors propose developing better visualization tools for scanpath networks and further exploring the relationship between working memory, executive control, and network metrics. They also suggest that this approach could be extended to other cognitive domains and research questions.

Summary updated

Abstract

Eye movements have been examined as an index of attention and comprehension during reading in the literature for over 30 years. Although eye-movement measurements are acknowledged as reliable indicators of readers' comprehension skill, few studies have analyzed eye-movement patterns using network science. In this study, we offer a new approach to analyze eye-movement data. Specifically, we recorded visual scanpaths when participants were reading expository science text, and used these to construct scanpath networks that reflect readers' processing of the text. Results showed that low ability and high ability readers' scanpath networks exhibited distinctive properties, which are reflected in different network metrics including density, centrality, small-worldness, transitivity, and global efficiency. Such patterns provide a new way to show how skilled readers, as compared with less skilled readers, process information more efficiently. Implications of our analyses are discussed in light of current theories of reading comprehension.
BibTeX
@inproceedings{ma2022from,
  author = {Xiaochuan Ma and Yikang Liu and Roy B. Clariana and Chanyuan Gu and Ping Li},
  title = {{From eye movements to scanpath networks: A method for studying individual differences in expository text reading}},
  booktitle = {Behavior},
  year = {2022},
  doi = {10.3758/s13428-022-01842-3},
}
Figure