Detection of Mild Cognitive Impairment from Language Markers with Crossmodal Augmentation

Guangliang Liu, Zhiyu Xue, Liang Zhan, Hiroko H. Dodge, Jiayu Zhou2022 · DOI

TL;DR

This paper introduces a novel "crossmodal augmentation" technique to improve the detection of Mild Cognitive Impairment (MCI) by combining language and brain imaging data. It addresses the challenge of limited datasets with both modalities by using imaging data to "augment" language features, significantly boosting MCI detection performance.

What Problem Does This Paper Address?

Early detection of Mild Cognitive Impairment (MCI) is crucial for Alzheimer's disease research and intervention. While brain imaging (like MRI) offers high sensitivity, its cost limits large-scale screening. Digital biomarkers, particularly language markers derived from speech, offer a more accessible alternative. However, language-based MCI detection models often suffer from limited data. The core problem this paper tackles is how to leverage the rich information from expensive imaging data to enhance the predictive power of more accessible language markers, especially when datasets with both types of data are scarce. This scarcity is a major bottleneck for traditional multi-modal analysis approaches.

Methods

The authors propose a "crossmodal augmentation" approach. The key idea is to train a model that learns the relationship between language and imaging features. This model, when given a subject's language markers, can generate "augmented" features that are informed by imaging data. This augmentation allows subjects who only have language data to benefit from the imaging information available in the dataset.

Specifically, they train a crossmodal augmentation model ($g_\omega$) using subjects who have both language and imaging data. This model learns to predict the similarity between language and imaging features. During inference, for a subject with only language data, this model uses their language features to construct an augmented feature vector by leveraging imaging features from other subjects. This augmented feature vector is then combined with the original language features to train a final MCI detection classifier ($f_\theta$). This approach is designed for high sample efficiency, making it suitable for datasets with limited overlap between modalities.

Datasets

The study utilizes data from the I-CONECT clinical trial.

  • Language Data: Conversational recordings from 40 subjects (20 MCI, 20 cognitively normal - NL). For each subject, 15 transcripts were used.
  • Imaging Data: Structural MRI (T1w) and diffusion MRI (dMRI) data from 43 subjects (26 MCI, 17 NL). Features extracted include cortical volume/thickness (74 ROIs) and brain connectome networks (85 ROIs).
  • Overlap: Crucially, only 16 subjects had both conversational recordings and MRI data available. This small overlap is the primary challenge addressed by the proposed method.

Key Insights and Ideas

  • Bridging the Modality Gap: The core insight is that even with limited subjects having both language and imaging data, a model can learn to "translate" or "augment" one modality using information from the other. This is a powerful way to overcome data scarcity in multi-modal learning.
  • Language as a Proxy for Imaging Insights: The augmented language features, while derived from language, incorporate information that mimics imaging features. This suggests that certain linguistic patterns might implicitly reflect underlying brain structural or functional changes.
  • Importance of dMRI: The analysis of top-ranked features revealed that diffusion MRI (dMRI) features, which were less prominent in unimodal imaging analysis, played a significant role in the crossmodal augmented model. This highlights the complementary information dMRI provides.
  • High Sample Efficiency: The proposed method's design allows for effective learning even with a small number of subjects possessing both modalities, which is a significant advantage over traditional multi-modal fusion techniques.

Key Results

  • Significant Performance Boost: The proposed "Ours-Lang-AugImg" method achieved a striking AUC of 0.973, significantly outperforming the unimodal MCI-Lang model (AUC 0.8) and slightly outperforming the unimodal MCI-Img model (AUC 0.969).
  • Inverse Augmentation Also Effective: Augmenting imaging markers with language information ("Ours-Img-AugLang") also yielded excellent results (AUC 0.98), though it's noted as less practical due to the inaccessibility of imaging markers.
  • Baseline Performance: Unimodal MCI-Lang achieved an AUC of 0.8, while MCI-Img achieved an AUC of 0.969.
  • Feature Importance: The top-ranked features in the augmented model were predominantly the "augmented" variables, underscoring the effectiveness of the crossmodal augmentation.

Limitations and Future Work

  • Anchor Marker Dependency: The augmented variables are constructed using a linear combination of given imaging markers. The authors suggest future work to study the impact of these "anchor" markers and explore refined choices.
  • Linear Assumption: The current augmentation relies on linear combinations. Future work could explore non-linear assumptions for potentially improved feature synthesis.
  • Convoluted Relationships: The complex relationship between language and imaging markers needs further investigation to guide future improvements.
  • Two Modalities Only: The current work focuses on two modalities. The authors believe their high sample-efficiency design can be extended to more than two modalities, which they plan to explore.
  • Generalizability: The method is proposed to be extendable to other clinical applications and can be adapted for privacy-preserving prediction in scenarios with missing modalities.

Summary updated

Abstract

Mild cognitive impairment is the prodromal stage of Alzheimer's disease. Its detection has been a critical task for establishing cohort studies and developing therapeutic interventions for Alzheimer's. Various types of markers have been developed for detection. For example, imaging markers from neuroimaging have shown great sensitivity, while its cost is still prohibitive for large-scale screening of early dementia. Recent advances from digital biomarkers, such as language markers, have provided an accessible and affordable alternative. While imaging markers give anatomical descriptions of the brain, language markers capture the behavior characteristics of early dementia subjects. Such differences suggest the benefits of auxiliary information from the imaging modality to improve the predictive power of unimodal predictive models based on language markers alone. However, one significant barrier to the joint analysis is that in typical cohorts, there are only very limited subjects that have both imaging and language modalities. To tackle this challenge, in this paper, we develop a novel crossmodal augmentation tool, which leverages auxiliary imaging information to improve the feature space of language markers so that a subject with only language markers can benefit from imaging information through the augmentation. Our experimental results show that the multi-modal predictive model trained with language markers and auxiliary imaging information significantly outperforms unimodal predictive models.
BibTeX
@misc{liu2022detection,
  author = {Guangliang Liu and Zhiyu Xue and Liang Zhan and Hiroko H. Dodge and Jiayu Zhou},
  title = {{Detection of Mild Cognitive Impairment from Language Markers with Crossmodal Augmentation}},
  year = {2022},
  doi = {10.1142/9789811270611\_0002},
}
Figure