Introduction
Autism Spectrum Disorder (ASD) affects approximately 1 in 59 children in the United States. Despite its prevalence, clinical screening remains a significant bottleneck due to the subjective nature of current diagnostic methods, which are time-consuming and require highly specialized personnel. While computer vision has been explored for gaze tracking in ASD screening, existing approaches often overlook temporal fixation dynamics or necessitate expensive, specialized equipment.
This research addresses three critical gaps: the scarcity of objective automated tools, the reliance on static heatmap analysis rather than temporal sequences, and the requirement for multi-modal data at inference time. The authors propose a framework that leverages "privileged information"—using one data modality during training to enhance the performance of a model processing a different modality during screening.
Methodology
The proposed framework integrates two deep neural network (DNN) models and a novel multi-modal distillation strategy.
Task-Specific Models
- Photo-Taking Task (Encoder-PT): Participants freely explore an environment and capture photographs. A ResNet-50 combined with a Long Short-Term Memory (LSTM) network analyzes these images. This task serves as an ecological proxy for social interaction, capturing preferences for objects over social subjects or atypical compositional styles.
- Image-Viewing Task (Encoder-IV): Participants view images while their eye movements are recorded. Unlike traditional static heatmap methods, this model uses an LSTM to process the temporal sequence of fixations, capturing the dynamic nature of visual attention.
Multi-Modal Distillation
The core innovation is a three-phase training process designed to transfer knowledge between modalities. Initially, models are trained independently. In the second phase, both models project features into a shared embedding space to learn commonalities between photo-taking and gaze patterns. Finally, in Phase III, this shared knowledge is distilled back into the individual models.
This approach allows the photo-taking model to benefit from gaze-pattern insights even when eye-tracking data is unavailable during the actual screening process.
Key Results
The framework demonstrates superior performance across both tasks, often surpassing human expert benchmarks.
Gaze-Based Screening
On the image-viewing task, the model achieved 100% sensitivity and 95% specificity, setting a new state-of-the-art for gaze-based ASD identification. The inclusion of temporal dynamics via LSTM proved crucial, as the order of fixations was found to be highly discriminative.
Photo-Taking Performance
The AI achieved 84% accuracy in identifying ASD from participant-taken photos. Notably, human experts (trained clinicians) only achieved 65% accuracy on the same dataset. The "Privileged Modality" distillation significantly boosted performance; the photo-taking model's accuracy improved from 76% (when trained independently) to 84% after distillation from the gaze-data shared space.
Discussion and Conclusion
The study highlights that temporal gaze patterns and ecological tasks like photo-taking provide rich, objective markers for ASD. The ability to transfer knowledge between different groups of participants (those who provided gaze data vs. those who took photos) makes this framework highly practical for clinical settings where multi-modal data collection for every patient is unfeasible.
Future work will focus on expanding the sample size beyond the current pilot cohort and exploring the use of standard webcams for gaze estimation to further reduce hardware barriers. Additionally, increasing the interpretability of the deep learning models will be essential for clinical adoption, helping practitioners understand the specific visual features driving the AI's classifications.
Summary updated