Assessing Map-Reading Skills Using Eye Tracking and Bayesian Structural Equation Modelling

Weihua Dong, Yuhao Jiang, Liangyu Zheng, Bing Liu, Liqiu MengSustainability 2018 · DOI

TL;DR

This paper introduces a novel approach to quantitatively assess map-reading skills by integrating eye-tracking data with Bayesian structural equation modeling. By analyzing eye movement patterns during various map-reading tasks, the study demonstrates that these patterns can effectively reflect a user's map-reading proficiency, offering a more cognitive perspective beyond traditional accuracy-based assessments.

What Problem Does This Paper Address?

Traditional methods for evaluating map-reading skills primarily rely on the accuracy of answers or response times. While these metrics provide insights into performance, they often fail to capture the underlying cognitive processes involved in understanding and interpreting maps. There's a notable gap in understanding how specific eye movement patterns relate to different facets of map-reading competence. This paper aims to bridge this gap by proposing a method that models the relationship between observable eye movements and latent map-reading skills, offering a more nuanced and cognitively-grounded assessment.

Methods

The core of this research lies in the application of Bayesian Structural Equation Modelling (SEM). The authors developed a model where map-reading skills are treated as a latent variable, influenced by three observable "measures": "measure of first fixation," "measure of processing," and "measure of search." These measures, in turn, are operationalized using specific eye-tracking metrics.

  • Eye Tracking: Participants' eye movements were recorded using a Tobii 120 eye tracker at a 60 Hz sample rate. Key metrics extracted included:
  • Measure of First Fixation: Time to first fixation and first-fixation duration within defined Areas of Interest (AOIs).
  • Measure of Processing: Percentage of total fixation duration and percentage of fixation count within AOIs.
  • Measure of Search: Saccade count and scan-path length (across the entire stimulus).
  • Bayesian Structural Equation Modelling: SEM was used to model the relationships between these eye movement metrics and the latent map-reading skill. A reflective model was employed, meaning the latent variables (measures) are influenced by the observed eye-tracking metrics. Bayesian imputation was used to handle non-numeric variables and estimate the model parameters.
  • Model Fit Evaluation: Standard fit indices like $\chi^2$/df, NFI, and RMSEA were used to assess how well the proposed model fit the data for each task.

What's novel here is the explicit modeling of the cognitive processes underlying map reading through eye movements, rather than just relying on outcome measures.

Datasets

The study utilized a dataset of 258 undergraduate students from Beijing Normal University. These participants completed five distinct map-reading tasks. No new benchmark datasets were introduced; rather, existing map-reading tasks were adapted and presented to the participants.

The five tasks were:

  1. Map Visualization: Choosing a map corresponding to verbal spatial information.
  2. Topology: Selecting a 3D view from a given spot and direction on a bird's-eye view.
  3. Navigation (Subway): Navigating a subway map using verbal instructions.
  4. Navigation (Campus): Navigating a campus map using verbal instructions.
  5. Spatial Association: Identifying the spatial correlation between a population map and a precipitation map.

Key Insights and Ideas

  • Eye Movements as Proxies for Cognitive Processes: The study strongly suggests that specific eye movement patterns are not just random behaviors but can serve as reliable indicators of cognitive processes involved in map reading. This opens up new avenues for understanding how people read maps, not just if they get the right answer.
  • "Measure of Processing" as a Strong Predictor: The "measure of processing" (fixation count and duration percentages) emerged as a particularly strong indicator of map-reading skill across most tasks. This implies that how deeply and for how long users engage with map elements is crucial for understanding.
  • Task-Dependent Relationships: The strength of the relationship between map-reading skill and eye movement metrics varied across the five tasks. This highlights that different types of map-reading skills might rely on different cognitive strategies, and thus manifest differently in eye movements.
  • Accuracy vs. Efficiency Trade-offs: While the model generally reflected both accuracy and efficiency, the "measure of processing" was weighted more heavily in the SEM, suggesting accuracy was a more dominant factor in the model's assessment of skill. This aligns with the idea that understanding the map correctly is often prioritized over speed.
  • Search Efficiency and Response Time: A strong negative correlation was found between the "measure of search" and response time, indicating that more efficient searching (fewer saccades, shorter scan paths) led to faster task completion.

Key Results

  • Model Fit: The proposed SEM model demonstrated good fit across all five map-reading tasks, indicating that the chosen eye movement metrics and their relationships to map-reading skills were well-supported by the data.
  • Path Coefficients:
    • Map-reading skill had a significant positive path coefficient to the "measure of processing" (ranging from 0.376 to 0.985 across tasks).
  • The relationship with the "measure of first fixation" was also positive but more variable (0.341 to 0.725).
  • The "measure of search" showed a mixed relationship, with positive coefficients for most tasks but a negative one for the spatial association task, suggesting that for some complex tasks, more search effort might be needed.
  • Imputed Scores: Participants who answered questions correctly consistently scored higher on the "map-reading skill" and "measure of processing" latent variables compared to those who answered incorrectly.
  • Correlation with Response Time: While map-reading skill scores showed weak correlations with response time, the "measure of search" scores exhibited strong negative correlations with response time, reinforcing the link between efficient visual search and task speed.

Limitations and Future Work

  • Limited Eye Movement Metrics: The authors acknowledge that only a selection of eye movement metrics was used. Further analysis with more metrics (e.g., mean fixation duration) and potentially new grouping methods could improve model accuracy.
  • Task Specificity: The study focused on five specific tasks. Future work could involve analyzing a wider range of map-reading tasks and potentially more stimuli per task to increase consistency and confidence in the model.
  • Participant Demographics: The study involved undergraduate students. Future research could explore gender effects and the impact of different academic backgrounds or courses on map-reading skills and eye movement patterns.
  • Validation: While the model showed good fit, further validation with independent datasets or different populations would strengthen its generalizability.
  • Dynamic Interactions: The current model is largely static. Future work could explore dynamic interactions and how eye movements evolve over time within a task.

Summary updated

Abstract

Map reading is an important skill for acquiring spatial information. Previous studies have mainly used results-based assessments to learn about map-reading skills. However, how to model the relationship between map-reading skills and eye movement metrics is not well documented. In this paper, we propose a novel method to assess map-reading skills using eye movement metrics and Bayesian structural equation modelling. We recruited 258 participants to complete five map-reading tasks, which included map visualization, topology, navigation, and spatial association. The results indicated that map-reading skills could be reflected in three selected eye movement metrics, namely, the measure of first fixation, the measure of processing, and the measure of search. The model fitted well for all five tasks, and the scores generated by the model reflected the accuracy and efficiency of the participants’ performance. This study might provide a new approach to facilitate the quantitative assessment of map-reading skills based on eye tracking.
BibTeX
@inproceedings{dong2018assessing,
  author = {Weihua Dong and Yuhao Jiang and Liangyu Zheng and Bing Liu and Liqiu Meng},
  title = {{Assessing Map-Reading Skills Using Eye Tracking and Bayesian Structural Equation Modelling}},
  booktitle = {Sustainability},
  year = {2018},
  doi = {10.3390/su10093050},
}
Figure