TL;DR
This paper introduces a novel approach to early dysgraphia screening by using a gamified visual perception test combined with eye-tracking. Machine learning models, particularly those leveraging game performance data, show promising results in predicting children at risk of dysgraphia, suggesting a more accessible and less stressful diagnostic tool.
What Problem Does This Paper Address?
Dysgraphia, a learning disability affecting handwriting, is often diagnosed late, after handwriting development is complete. This delay hinders timely intervention and can lead to significant psychological distress for affected children. Current diagnostic methods can be subjective, time-consuming, and administered when the problem is already established. The paper addresses the need for an earlier, more objective, and less stressful screening method by investigating the link between visual perception skills and handwriting difficulties. Specifically, it explores whether gamified visual perception tasks, coupled with eye-tracking data, can serve as early indicators of dysgraphia.
Methods
The researchers developed a 2D, single-player serious game designed for primary school children. This game incorporates gamified versions of standard clinical visual perception tests (e.g., form constancy, masked form constancy, copying, tracing, figure-ground perception, visual closure) across three difficulty levels. Participants played these games while wearing an eye-tracker to record their gaze patterns and fixations.
To predict dysgraphia risk, five machine learning models (Gaussian Naive Bayes, Random Forest, Support Vector Machine, CatBoost) were trained and evaluated. The models used combinations of features derived from:
- Game Performance: Time to completion, errors.
- Eye-Tracking Data: Gaze positions, fixations, indecisiveness (number of gaze changes between elements), attention (percentage of gaze on target).
- Drawing Data: Features related to tract continuity, length, and spatial organization from copying and tracing tasks.
A Nested Cross-Validation approach was employed to ensure robust evaluation on a small dataset. SHAP (SHapley Additive exPlanations) was used to interpret the models and identify the most influential features.
Datasets
The study utilized a dataset of 53 primary school children (aged 7.84 ± 0.90).
- Training/Testing Data: This dataset was derived from the children's performance in the serious game (game performance, eye-tracking, and drawing data) and their scores on the BVSCO-2 (Batteria per la Valutazione della Scrittura e della Competenza Ortografica) writing speed test. The BVSCO-2 results were dichotomized into "at-risk" and "not-at-risk" categories to serve as the ground truth for the machine learning models.
- Normative Data: The BVSCO-2 test itself relies on normative data obtained from over 350 children.
No new benchmark datasets were introduced; rather, existing clinical assessments were adapted and integrated into a novel gamified framework.
Key Insights and Ideas
- Visual Perception as a Predictor: The study strongly supports the hypothesis that visual perception abilities are significantly linked to handwriting performance and can serve as an early indicator of dysgraphia.
- Gamification for Engagement: Using serious games for assessment proved highly effective, with a reported usability score of 89.4 ± 9.6, indicating excellent acceptance and engagement among children. This suggests gamification can mitigate the stress and anxiety often associated with traditional testing.
- Eye-Tracking for Subtle Clues: Eye-tracking data, particularly measures of indecisiveness and attention, provided valuable insights into cognitive processes underlying visual perception and their relation to handwriting risk.
- Game Performance Dominance: Interestingly, game performance metrics alone (time and errors) emerged as the strongest predictors of dysgraphia risk, achieving an F1 score of 0.71 on the test set. This highlights the potential of a game-based assessment without the need for more complex equipment like eye-trackers.
- Drawing Features Matter: While game performance was strongest, drawing features, especially those related to tract discontinuity and spatial organization in copying tasks, also contributed significantly to predicting risk.
- Holistic Approach: The most effective models incorporated a combination of features, suggesting that visual perception, eye movements, and drawing skills collectively contribute to handwriting proficiency.
Key Results
- Machine Learning Performance: The CatBoost classifier, using game performance data, achieved an F1 score of 0.77 on the training set and 0.71 on the test set, demonstrating good predictive power for dysgraphia risk.
- Usability: The serious game achieved an excellent usability score of 89.4 ± 9.6, indicating high acceptance by the child participants.
- Feature Importance: SHAP analysis revealed that Visual Closure, Masked Form Constancy, and Figure-Ground Perception games were particularly influential in predicting risk when using game performance data. For eye-tracking, indecisiveness (number of gaze changes) and average fixation duration were key indicators. For drawing, tract discontinuity and spatial organization were significant.
- Gender Differences: A slight but statistically significant difference was observed, with males showing slightly worse performances than females in the BVSCO-2, aligning with historical observations.
Limitations and Future Work
- Sample Size: The study acknowledges a relatively small sample size (n=53), which can limit the generalizability of the findings. Future work should extend the research to a larger and more diverse population.
- Eye-Tracker Challenges: The head-mounted eye-tracker presented some usability challenges, including discomfort and potential for chaotic data due to head movements. Future research could explore display-mounted eye trackers or alternative eye-tracking solutions.
- Apple Pencil Data: The study did not fully integrate Apple Pencil data (e.g., pressure, tremor) into the predictive models, despite acknowledging its potential influence on handwriting. Future work could incorporate these quantitative measurements.
- Narrowing Scope: The authors suggest that future research could benefit from narrowing the focus to specific visual perception abilities that demonstrated stronger predictive power.
- Cost of Technology: While the game performance model showed strong results, the reliance on eye-trackers and iPads might still be a barrier for widespread adoption in schools. Future work could focus on developing standalone game-based screening tools that are more accessible.
Summary updated