Identifying Progression-Specific Alzheimer’s Subtypes Using Multimodal Transformer

Diego Machado Reyes, Hanqing Chao, Juergen Hahn, Li Shen, Pingkun Yan, for the Alzheimer’s Disease Neuroimaging Initiative

TL;DR

This paper introduces Tri-COAT, a novel multimodal deep learning framework that uses imaging, genetics, and clinical data to identify distinct progression subtypes of Alzheimer's Disease (AD) at an early stage. By employing a tri-modal co-attention mechanism, Tri-COAT not only achieves superior classification performance but also provides interpretable insights into the relationships between different biomarkers.

What Problem Does This Paper Address?

Alzheimer's Disease (AD) is a complex and heterogeneous neurodegenerative disorder, making effective treatment and personalized medicine challenging. Current treatments primarily focus on slowing progression, but their efficacy is hampered by the disease's varied nature. A critical gap exists in accurately identifying distinct AD subtypes, especially in the early, asymptomatic, or prodromal stages. Existing data-driven approaches often rely on single data modalities, lack explainability, and struggle with early-stage prediction. This paper aims to bridge this gap by developing a multimodal approach that leverages early-stage indicators to classify AD patients into progression-specific subtypes, offering both predictive power and interpretability.

Methods

The core innovation of this work is the Tri-COAT (Tri-Modal Co-Attention) framework. This model is designed to effectively fuse information from three distinct modalities: brain imaging (MRI), genetics (SNPs), and clinical assessments.

The framework consists of two main parts:

  1. Single-Modality Encoding: Each data modality is processed by an independent transformer encoder.
  2. Imaging: MRI data, specifically quantitative traits derived from T1-weighted scans (cortical volume, thickness, surface area), are tokenized by region of interest (ROI) and fed into a transformer.
  3. Genetics: Single Nucleotide Polymorphism (SNP) data, augmented with information like allele dosage, odds ratio, and intergenic region status, are tokenized and processed by another transformer. Chromosome embeddings are also incorporated for location context.
  4. Clinical: Clinical assessment data, which are already relatively low-dimensional, are directly tokenized and fed into a transformer.
  5. Tri-Modal Co-Attention: This is the novel component. It uses the learned representations from the single-modality encoders to explicitly learn cross-modal feature relationships. Specifically, imaging and genetic features act as "query" and "key/value" matrices to modulate the "clinical" branch. The intuition is that clinical data most directly reflects the disease phenotype, and imaging/genetics enrich this representation by highlighting shared biological mechanisms. This co-attention mechanism allows the model to weigh features based on their inter-modal relevance.

Finally, the fused representation is passed through a Multilayer Perceptron (MLP) for disease subtype classification.

Datasets

The study utilizes data from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database. ADNI is a well-established, large-scale dataset that includes a wide range of modalities such as MRI, PET, genetics, cognitive tests, CSF, and blood biomarkers. For this study, longitudinal data were selected for up to two years of progression.

Subjects were clustered into three progression-specific subtypes (slow, intermediate, and fast) based on their Mini-Mental State Examination (MMSE) score decline over time. The dataset comprised:

  • Slow progressing: 177 participants
  • Intermediate progressing: 302 participants
  • Fast progressing: 15 participants

The models were trained and evaluated using a 10-fold stratified cross-testing approach.

Key Insights and Ideas

  • Early-stage multimodal fusion is crucial for AD subtyping: The heterogeneity of AD means that relying on single data sources or late-stage symptoms is insufficient for accurate subtyping. Combining early indicators from imaging, genetics, and clinical assessments offers a more comprehensive view.
  • Co-attention unlocks cross-modal insights: The Tri-COAT mechanism's ability to explicitly learn how features from different modalities interact is key. This goes beyond simple concatenation or late-stage aggregation, allowing the model to understand how genetic predispositions or imaging changes manifest in clinical symptoms.
  • Clinical data as the "subject" enriches understanding: The paper frames clinical data as the core "subject" of the disease phenotype, with imaging and genetics acting as "adjectives" and "adverbs" that provide richer context. This highlights the importance of clinical assessments while acknowledging the complementary power of other modalities.
  • Interpretability through attention: The co-attention mechanism not only improves performance but also provides a window into why the model makes certain predictions, identifying specific biomarker associations that are biologically plausible.

Key Results

  • Superior Classification Performance: Tri-COAT achieved a mean Area Under the Receiver Operating Characteristic Curve (AUROC) of 0.734 ± 0.076 across all test sets in the 10-fold cross-testing framework. This significantly outperformed baseline models, including single-modality transformer models, traditional machine learning models (SVM, RF), and a stage-wise fusion deep learning model.
  • Multimodal Advantage: All multimodal approaches, including Tri-COAT, outperformed their single-modality counterparts, reinforcing the value of integrating diverse data sources for AD classification.
  • Biomarker Association Insights: The attention visualization revealed key cross-modal associations. For instance, it identified a link between the Trails Making Test B score (TRABSCORE) in clinical data, specific brain regions (temporal gyrus) in imaging, and the CD2AP gene in genetics. This finding aligns with existing biological knowledge about AD mechanisms.
  • Fusion Strategy Matters: Tri-COAT significantly outperformed early and late fusion strategies, demonstrating that intermediate fusion with co-attention is more effective for handling heterogeneous AD data.

Limitations and Future Work

  • Generalizability: The authors acknowledge that the generalizability of the learned features to other datasets remains to be tested.
  • Future Directions:
    • Extension to other neurodegenerative diseases: The framework could be adapted for diseases like Parkinson's.
  • Incorporation of additional modalities: Future work could include PET imaging (for fluid biomarkers) and transcriptomic data (to bridge genotype and endophenotypes), potentially leading to a deeper understanding of underlying mechanisms and new therapeutic targets.

Summary updated

Abstract

Alzheimer's disease (AD) is the most prevalent neurodegenerative disease, yet its current treatments are limited to stopping disease progression. Moreover, the effectiveness of these treatments remains uncertain due to the heterogeneity of the disease. Therefore, it is essential to identify disease subtypes at a very early stage. Current data-driven approaches can be used to classify subtypes during later stages of AD or related disorders, but making predictions in the asymptomatic or prodromal stage is challenging. Furthermore, the classifications of most existing models lack explainability, and these models rely solely on a single modality for assessment, limiting the scope of their analysis. Thus, we propose a multimodal framework that utilizes early-stage indicators, including imaging, genetics, and clinical assessments, to classify AD patients into progression-specific subtypes at an early stage. In our framework, we introduce a tri-modal co-attention mechanism (Tri-COAT) to explicitly capture cross-modal feature associations. Data from the Alzheimer's Disease Neuroimaging Initiative (ADNI) (slow progressing = 177, intermediate = 302, and fast = 15) were used to train and evaluate Tri-COAT using a 10-fold stratified cross-testing approach. Our proposed model outperforms baseline models and sheds light on essential associations across multimodal features supported by known biological mechanisms. The multimodal design behind Tri-COAT allows it to achieve the highest classification area under the receiver operating characteristic curve while simultaneously providing interpretability to the model predictions through the co-attention mechanism.
BibTeX
@article{reyes2024identifying,
  author = {Diego Machado Reyes and Hanqing Chao and Juergen Hahn and Li Shen and Pingkun Yan and for the Alzheimer’s Disease Neuroimaging Initiative},
  title = {{Identifying Progression-Specific Alzheimer’s Subtypes Using Multimodal Transformer}},
  journal = {Journal of Personalized Medicine},
  year = {2024},
  doi = {10.3390/jpm14040421},
}
Figure