Oral-Motor and Lexical Diversity During Naturalistic Conversations in Adults with Autism Spectrum Disorder

Julia Parish-Morris, Evangelos Sariyanidi, Casey Zampella, G. Keith Bartley, Emily Ferguson, Ashley A. Pallathra, Leila Bateman, Samantha Plate, Meredith Cola, Juhi Pandey, Edward S. Brodkin, Robert T. Schultz, Birkan TunçNAACL 2018

TL;DR

This study introduces a novel computational approach to quantify "restrictedness" in adults with Autism Spectrum Disorder (ASD) by analyzing both their spoken language and facial movements during natural conversations. The findings reveal that individuals with ASD exhibit significantly less diversity in both their word choices and mouth movements, suggesting a cross-domain manifestation of restricted behavior.

What Problem Does This Paper Address?

Diagnosing and understanding Autism Spectrum Disorder (ASD) is challenging due to its extreme heterogeneity. Traditional assessment methods are often time-consuming, expensive, and may not fully capture the nuances of ASD in naturalistic settings. While research has identified restricted and repetitive behaviors as a core feature of ASD, quantifying this "restrictedness" across different behavioral domains, especially in adults during natural conversations, remains an open challenge. This paper aims to address this by developing and applying computational methods to identify and measure this restricted dimension in both linguistic and oral-motor behaviors.

Methods

The study employs a multi-modal computational approach combining natural language processing (NLP) and computer vision.

  • Lexical Diversity: This was quantified using Shannon entropy, a measure from information theory that captures the diversity or unpredictability of a data source. For language, this translates to the variety of words used by a participant. The "qdap" R package was used to calculate lexical entropy.
  • Oral-Motor Diversity: This was measured by analyzing facial movements, specifically mouth movements, using computer vision. A pipeline involving face detection, registration, and facial landmark localization (using OpenFace) was employed. Facial movements were then quantified using the "Facial Bases" method, which breaks down complex movements into 180 basis functions, 60 of which are specific to mouth movements. Shannon entropy was then applied to the activation patterns of these facial bases to quantify oral-motor diversity.
  • Statistical Analysis: Linear mixed-effects models were used to account for the nested design (confederates within participants). Ordinary linear regression models were used when random effects did not significantly improve model fit. Covariates such as age, IQ, sex, speech length, and head movement were included in the models to control for potential confounding factors.

Datasets

The study utilized data from 44 adult participants (17 with ASD, 27 neurotypical controls) engaged in naturalistic "get to know you" conversations. The conversations were recorded using a custom-built "TreeCam" device that captured both audio and video. No new benchmark datasets were introduced; rather, existing computational tools and established assessment protocols (like the CASS) were adapted and integrated.

Key Insights and Ideas

  • Cross-Domain Restriction: The most significant insight is the demonstration that restricted behavior in ASD is not confined to a single modality but appears to manifest across both linguistic and oral-motor domains during natural conversations.
  • Entropy as a Marker: Shannon entropy proves to be a valuable metric for quantifying diversity and, by extension, "restrictedness" in complex human behaviors like speech and facial movements.
  • Complementary Information: The lack of significant correlation between lexical and oral-motor diversity suggests that these two measures capture distinct, yet complementary, aspects of restricted behavior in ASD. This implies that a comprehensive understanding requires assessing multiple behavioral channels.
  • Interaction of Word Count and Diversity: The study found an interesting interaction effect where the relationship between word count and lexical diversity differed between ASD and neurotypical groups. While TD individuals showed a steeper increase in diversity with more words, ASD individuals did not reach the same levels of diversity even with increased word production.

Key Results

  • Lexical Diversity: Participants with ASD showed significantly reduced lexical diversity compared to neurotypical controls (Cohen's d=0.82). This difference remained significant even after accounting for age, IQ, and gender.
  • Oral-Motor Diversity: A significant decrease in mouth movement diversity was observed in the ASD group (Cohen's d=1.0). This finding held even when controlling for head movement and speech length.
  • Conversational Differences: Participants with ASD produced fewer words and longer pauses during conversations compared to the TD group.
  • Correlation: Lexical and oral-motor diversity values were not significantly correlated within the ASD group, the TD group, or the overall sample.

Limitations and Future Work

  • Sample Size: The study acknowledges its relatively small sample size (n=44), which limits the complexity of the statistical models that can be employed.
  • Intervention Effects: The ASD participants had recently completed an intensive social skills intervention. The authors suggest that the observed differences might even underestimate the magnitude of restrictedness in untreated individuals.
  • Future Directions:
    • Investigating the relationship between reduced linguistic diversity and perceived social communication ability by gathering confederate ratings.
  • Exploring how word category (e.g., function vs. content words) influences the observed linguistic diversity differences.
  • Examining the potential for oral-motor impairments to contribute to reduced phonological diversity.
  • Developing objective methods to quantify the integration of language and facial expressions during conversations.
  • Exploring the relationship between reduced behavioral diversity and clinical phenotype to predict individual variability in ASD.
  • Utilizing these computational methods to identify and track quantifiable treatment targets for restricted/repetitive behaviors, advancing precision medicine for autism.

Summary updated

BibTeX
@inproceedings{parishmorris2018oral,
  author = {Julia Parish-Morris and Evangelos Sariyanidi and Casey Zampella and G. Keith Bartley and Emily Ferguson and Ashley A. Pallathra and Leila Bateman and Samantha Plate and Meredith Cola and Juhi Pandey and Edward S. Brodkin and Robert T. Schultz and Birkan Tunç},
  title = {{Oral-Motor and Lexical Diversity During Naturalistic Conversations in Adults with Autism Spectrum Disorder}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2018},
}
Figure