Identification and characterization of learning weakness from drawing analysis at the pre-literacy stage

Linda Greta Dui, Eugenio Lomurno, Francesca Lunardini, Cristiano Termine, Alessandro Campi, Matteo Matteucci, Simona FerranteScientific 2022 · DOI

TL;DR

This paper introduces a novel method for early detection of handwriting learning difficulties in pre-literacy children by analyzing their drawings on a tablet, rather than relying on traditional handwriting assessments. A machine learning model achieved promising accuracy in identifying at-risk children, paving the way for targeted early interventions.

What Problem Does This Paper Address?

Handwriting difficulties, or dysgraphia, can have significant long-term negative impacts on a child's academic and personal development. Early intervention is crucial, but current diagnostic methods often require children to have already learned to write, and in-person assessments can be inaccessible. This leaves a critical gap in identifying and addressing potential issues at the pre-literacy stage, before formal writing instruction begins. The authors aim to bridge this gap by developing a method that can objectively assess graphical abilities in young children through drawing analysis, enabling earlier and more accessible screening.

Methods

The core of this research lies in a tablet-based application called "Play-Draw-Write." This app engages children in drawing various symbols and shapes at different sizes and complexities, mimicking aspects of handwriting that are known to be affected in dysgraphia (e.g., isochrony, homothety, speed-accuracy trade-off). Crucially, the app captures detailed kinematic and pressure data from the stylus.

The researchers extracted a rich set of features from these drawings, categorizing them into areas like Kinematics, Fluidity, Angles, Pressure, and Errors. To quantify a child's deviation from typical performance, they developed an "outlier score" (O-score). This score aggregates deviations across various features, indicating potential graphical weaknesses.

For risk detection, they employed a 1-dimensional convolutional neural network (CNN) called "LearNet." This model was trained on the extracted features to classify children as either at-risk (R) or not-at-risk (NR) for handwriting delay, based on teacher evaluations. To understand why the model made its predictions, they utilized SHapley Additive exPlanations (SHAP) to identify the most influential features.

Datasets

The study involved 241 kindergarten children. These children completed the Play-Draw-Write games, generating a dataset of their drawings and associated kinematic/pressure data. This data was used to train and evaluate the machine learning models. Teacher evaluations, based on a 15-item checklist, served as the ground truth for identifying children at risk of dysgraphia.

Key Insights and Ideas

  • Drawing as a Precursor to Writing: The study strongly suggests that drawing patterns at the pre-literacy stage can reveal underlying difficulties that will manifest in handwriting later on. This validates the idea of using drawing as a diagnostic tool for future writing challenges.
  • Objective Measurement is Key: Relying on objective, quantifiable features extracted from tablet drawings offers a significant advantage over subjective teacher observations, which can be inconsistent and experience-dependent.
  • The Power of "Outlier" Detection: The O-score effectively captures deviations from typical performance, highlighting children who may be struggling even if they don't exhibit obvious writing problems yet. This is a novel way to operationalize "risk" in this context.
  • Machine Learning for Early Detection: The successful application of a CNN (LearNet) demonstrates the potential of AI to automate and scale early screening for handwriting difficulties, making it more accessible.
  • Explainable AI for Targeted Intervention: Using SHAP to understand feature importance is crucial. It allows educators to pinpoint specific weaknesses (e.g., pressure control, speed consistency) that need attention, enabling tailored training rather than generic approaches.

Key Results

  • The machine learning model (LearNet) achieved a sensitivity of 75.0% and a specificity of 76.5% in classifying children at risk of handwriting delay.
  • The "Copy Game-Sequence" proved to be the most effective game for distinguishing between at-risk and not-at-risk children in unsupervised analysis, likely due to its demands on visual discrimination, working memory, and spatial awareness.
  • The study found that children identified as "at risk" by teachers tended to have significantly higher O-scores, indicating more pronounced deviations in their drawing features.
  • Feature importance analysis revealed that no single feature was overwhelmingly dominant, suggesting that a combination of features is necessary for accurate prediction, and that the model learned complex patterns. Frequency-related features were highlighted as particularly important.

Limitations and Future Work

The authors acknowledge that the "risk label" itself, derived from teacher checklists, can have inherent subjectivity and uncertainty. Future work will involve a two-year follow-up to track children's actual handwriting development and refine the risk assessment. They also plan to explore the feasibility of periodic self-assessment and automated risk evaluation in home settings as part of the European ESSENCE project. Additionally, they suggest that older children could be tested on the "Tunnel Game" to better understand its potential for assessing speed-accuracy trade-off development. The language-agnostic nature of the drawing-based approach is a significant advantage, but further validation across different linguistic backgrounds would be beneficial.

Summary updated

Abstract

Handwriting learning delays should be addressed early to prevent their exacerbation and long-lasting consequences on whole children's lives. Ideally, proper training should start even before learning how to write. This work presents a novel method to disclose potential handwriting problems, from a pre-literacy stage, based on drawings instead of words production analysis. Two hundred forty-one kindergartners drew on a tablet, and we computed features known to be distinctive of poor handwriting from symbols drawings. We verified that abnormal features patterns reflected abnormal drawings, and found correspondence in experts' evaluation of the potential risk of developing a learning delay in the graphical sphere. A machine learning model was able to discriminate with 0.75 sensitivity and 0.76 specificity children at risk. Finally, we explained why children were considered at risk by the algorithms to inform teachers on the specific weaknesses that need training. Thanks to this system, early intervention to train specific learning delays will be finally possible.
BibTeX
@inproceedings{dui2022identification,
  author = {Linda Greta Dui and Eugenio Lomurno and Francesca Lunardini and Cristiano Termine and Alessandro Campi and Matteo Matteucci and Simona Ferrante},
  title = {{Identification and characterization of learning weakness from drawing analysis at the pre-literacy stage}},
  booktitle = {Scientific},
  year = {2022},
  doi = {10.1038/s41598-022-26038-9},
}
Figure