TL;DR
This paper introduces novel computational methods to quantify interpersonal synchrony in interactions between children and clinicians, specifically focusing on children with Autism Spectrum Disorder (ASD). By analyzing vocal and linguistic cues, they demonstrate that these synchrony measures can effectively distinguish between children with and without ASD, offering a promising avenue for objective assessment and personalized interventions.
What Problem Does This Paper Address?
Autism Spectrum Disorder (ASD) is characterized by significant differences in social communication and interaction. While existing research has explored synchrony in specific modalities like vocal prosody, a comprehensive understanding of how synchrony manifests across different communication channels (vocal and linguistic) in ASD interactions is lacking. This gap hinders the development of objective, data-driven tools for diagnosis, monitoring, and personalized interventions. The paper aims to address this by developing and evaluating methods to quantify interpersonal synchrony across multiple modalities, hypothesizing that these measures can reveal subtle differences in interaction patterns associated with ASD.
Methods
The researchers propose and evaluate three distinct measures of interpersonal synchrony:
- Dynamic Time Warping Distance (DTWD): This classic method is adapted to measure the similarity between acoustic-prosodic features (pitch, intensity, jitter, shimmer) and spectral features (MFCCs) extracted from consecutive speaker turns. DTWD aligns time series of potentially different lengths to find the minimum "warping distance," with larger distances indicating less coordination.
- Squared Cosine Distance of Complexity Measure (SCDC): This novel approach quantifies synchrony by comparing the complexity of vocal and spectral features between turns. Complexity is measured using sample entropy, which reflects the amount of new information in a temporal sequence. A smaller difference in complexity between turns suggests higher synchrony.
- Word Mover's Distance (WMD): For lexical features, WMD is used to capture semantic similarity between turns. It measures the minimum "distance" that word embeddings (contextual representations from BERT) need to travel to transform one turn's semantic representation into another's. Lower WMD is hypothesized to indicate greater synchrony.
The core innovation lies in applying these measures across different modalities and then fusing them to capture a more holistic view of interaction synchrony.
Datasets
The study utilizes data from the Autism Diagnostic Observation Schedule (ADOS-2) instrument. Specifically, they focus on Module 3 data from verbally fluent children, comprising interactions from two subtasks: "Emotions" and "Social difficulties and annoyance." The dataset includes recordings from 165 children (86 with ASD, 79 without ASD) collected from two clinics: the University of Michigan Autism and Communication Disorders Center (UMACC) and the Cincinnati Children's Medical Center (CCHMC). For analysis, sessions with fewer than 10 turns were excluded.
Key Insights and Ideas
- Multimodal Synchrony is Key: Fusing synchrony measures from vocal (prosodic, spectral) and linguistic (lexical) modalities significantly improves the ability to distinguish between children with and without ASD compared to using individual modalities alone. This highlights that synchrony is a complex phenomenon that spans across different communication channels.
- Complementary Information: The study demonstrates that vocal and linguistic synchrony features capture complementary information. This suggests that a comprehensive understanding of interaction dynamics requires considering multiple communication aspects.
- Age and Gender Influence Synchrony: The analysis reveals that age and gender significantly affect classification performance based on synchrony measures. This points to developmental and sex-based differences in social interaction patterns that are reflected in synchrony.
- Emotion Subtask Shows Less Synchrony: Across all measures, the "Emotion" subtask consistently shows lower synchrony for children with ASD compared to those without ASD, and also generally lower synchrony than the "Social" subtask for all children. This suggests that emotional expression and interaction might be particularly challenging in terms of synchrony for individuals with ASD.
Key Results
- Classification Performance: The proposed synchrony measures, particularly when fused, show promising results in classifying children with and without ASD. While specific F1 scores vary by classifier and feature set, fused features generally yield higher performance. For instance, prosodic and spectral features combined showed the best performance in some fused-feature experiments.
- DTWD Outperforms SCDC for Prosodic/Spectral Features: In the case of spectral and prosodic features, DTWD generally yielded better classification performance than SCDC.
- Age and Gender Impact: A two-way ANOVA indicated that age and gender significantly affect the F1 scores for ASD/non-ASD classification (p value = 0.000196). Notably, female children in the oldest age group showed improved classification performance.
- ASD Exhibits Less Synchrony: Figure 3 visually demonstrates that children with ASD generally exhibit higher mean values for the proposed synchrony measures across both subtasks, indicating less synchrony compared to children without ASD.
Limitations and Future Work
- Limitations: The paper acknowledges that the current work focuses on specific subtasks of the ADOS-2. The generalizability of these findings to other interaction contexts or broader ASD populations may require further investigation. While WMD has been validated in prior work, its application here is primarily to complement the novel DTWD and SCDC measures.
- Future Work: The authors suggest exploring more data-driven approaches, such as deep neural networks, to quantify synchrony. They believe that neural networks can learn non-linear mappings between features and coordination measures, potentially leading to more sophisticated representations of synchrony and improved characterization of ASD. Further research into the interplay between different modalities and their contribution to synchrony is also a promising direction.
Summary updated