Multimodal normative modeling in Alzheimer’s Disease with introspective variational autoencoders

Sayantan Kumar, Peijie Qiu, Braden Yang, Abdalla Bani, Philip Payne, Aristeidis SotirasbioRxiv 2024 · DOI

Abstract

Normative models in neuroimaging learn patterns of healthy brain distributions to identify deviations in disease subjects, such as those with Alzheimer’s Disease (AD). This study addresses two key limitations of variational autoencoder (VAE)-based normative models: (1) VAEs often struggle to accurately model healthy control distributions, resulting in high reconstruction errors and false positives, and (2) traditional multimodal aggregation methods, like Product-of-Experts (PoE) and Mixture-of-Experts (MoE), can produce uninformative latent representations. To overcome these challenges, we developed a multimodal introspective VAE that enhances normative modeling by achieving more precise representations of healthy anatomy in both the latent space and reconstructions. Additionally, we implemented a Mixture-of-Product-of-Experts (MOPOE) approach, leveraging the strengths of PoE and MoE to efficiently aggregate multimodal information and improve abnormality detection in the latent space. Using multimodal neuroimaging biomarkers from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset, our proposed multimodal introspective VAE demonstrated superior reconstruction of healthy controls and outperformed baseline methods in detecting outliers. Deviations calculated in the aggregated latent space effectively integrated complementary information from multiple modalities, leading to higher likelihood ratios. The model exhibited strong performance in Out-of-Distribution (OOD) detection, achieving clear separation between control and disease cohorts. Additionally, Z-score deviations in specific latent dimensions were mapped to feature-space abnormalities, enabling interpretable identification of brain regions associated with AD pathology.

Introduction

In neuroimaging, normative modeling serves as a powerful framework for identifying individual deviations from a healthy population distribution. These deviations often signal the onset of neurodegenerative diseases like Alzheimer’s Disease (AD). However, current Variational Autoencoder (VAE) models face significant challenges. First, they often suffer from high false-positive rates because they cannot perfectly reconstruct even healthy scans, leading to "reconstruction noise" being misinterpreted as pathology. Second, integrating multiple data modalities—such as structural MRI and Amyloid PET—remains difficult. Traditional methods like Product-of-Experts (PoE) can be overly sensitive to a single modality, while Mixture-of-Experts (MoE) may fail to create a truly synergistic joint representation.

This research introduces the multimodal soft-introspective VAE (mmSIVAE), a novel architecture designed to define "normal" brain aging with higher precision. By combining adversarial training with a Mixture-of-Product-of-Experts (MOPOE) approach, the model improves the detection of subtle AD signatures and provides interpretable mappings back to brain anatomy.

Methods

The core innovation of the mmSIVAE framework is its "introspective" training strategy. Unlike standard VAEs that focus solely on compression and reconstruction, the introspective VAE treats the encoder as a critic in a min-max game, similar to a Generative Adversarial Network (GAN). The encoder is trained to distinguish between real healthy scans and the "fake" reconstructions produced by the decoder. This adversarial "push-pull" mechanism forces the model to learn a tighter, more accurate representation of the healthy control distribution.

Schematic of the mmSIVAE training architecture, highlighting the adversarial 'push-pull' between encoders and decoders to refine the latent space.

To handle multimodal data, the researchers implemented a Mixture-of-Product-of-Experts (MOPOE) hierarchy. This method calculates the Product-of-Experts for every possible subset of modalities and then aggregates these using a Mixture-of-Experts. This ensures the model remains robust to missing data and prevents any single modality from dominating the joint latent representation.

The study utilized data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI). The training set consisted of 248 cognitively unimpaired (CU) subjects with no amyloid pathology. Evaluation was performed on a holdout group of 48 CU subjects and a large disease cohort of 726 individuals across the Alzheimer’s Disease Spectrum (ADS), using regional brain volumes from T1-weighted MRI and Standardized Uptake Value Ratios (SUVR) from AV45 Amyloid PET.

Results

The mmSIVAE framework demonstrated significant improvements over baseline models (such as mmJSD and JMVAE) across several metrics:

  • Reconstruction Accuracy: The model achieved the lowest reconstruction errors for both MRI and PET data, indicating a superior ability to model the healthy distribution.
  • Outlier Detection: By calculating Mahalanobis distance in the latent space ($D_{ml}$), the model achieved a likelihood ratio of 10.2 for outlier detection, outperforming traditional feature-space reconstruction error methods.
  • Distributional Separation: The Earth Mover’s Distance (EMD) between healthy controls and AD patients was highest for mmSIVAE (34.043), signifying a clearer separation of disease cohorts in the latent space.
  • Clinical Relevance: Latent deviations showed strong correlations with clinical scores in memory, executive function, and language, suggesting the model captures biologically meaningful patterns of decline.

Furthermore, the model provides high interpretability. By isolating latent dimensions with high Z-scores and passing them through the decoder, the researchers could map "abnormal" latent signals back to specific brain regions like the hippocampus or precuneus, which are known hubs for AD pathology.

Discussion and Conclusion

The mmSIVAE framework addresses critical gaps in normative modeling by refining the latent representation of healthy anatomy through adversarial training. The MOPOE integration successfully balances multimodal information, leading to more sensitive and specific detection of AD-related abnormalities.

Future work will aim to expand the model to include additional modalities like Tau PET or CSF biomarkers to align with the ATN (Amyloid, Tau, Neurodegeneration) framework. Additionally, applying this model to longitudinal data could enable the prediction of individual disease trajectories, moving from cross-sectional detection to proactive clinical forecasting. While hyperparameter tuning remains a challenge, the results establish mmSIVAE as a robust tool for multimodal neuroimaging analysis.

Summary updated

Abstract

Abstract Normative models in neuroimaging learn patterns of healthy brain distributions to identify deviations in disease subjects, such as those with Alzheimer’s Disease (AD). This study addresses two key limitations of variational autoencoder (VAE)-based normative models: (1) VAEs often struggle to accurately model healthy control distributions, resulting in high reconstruction errors and false positives, and (2) traditional multimodal aggregation methods, like Product-of-Experts (PoE) and Mixture-of-Experts (MoE), can produce uninformative latent representations. To overcome these challenges, we developed a multimodal introspective VAE that enhances normative modeling by achieving more precise representations of healthy anatomy in both the latent space and reconstructions. Additionally, we implemented a Mixture-of-Product-of-Experts (MOPOE) approach, leveraging the strengths of PoE and MoE to efficiently aggregate multimodal information and improve abnormality detection in the latent space. Using multimodal neuroimaging biomarkers from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset, our proposed multimodal introspective VAE demonstrated superior reconstruction of healthy controls and outperformed baseline methods in detecting outliers. Deviations calculated in the aggregated latent space effectively integrated complementary information from multiple modalities, leading to higher likelihood ratios. The model exhibited strong performance in Out-of-Distribution (OOD) detection, achieving clear separation between control and disease cohorts. Additionally, Z-score deviations in specific latent dimensions were mapped to feature-space abnormalities, enabling interpretable identification of brain regions associated with AD pathology.
BibTeX
@misc{kumar2024multimodal,
  author = {Sayantan Kumar and Peijie Qiu and Braden Yang and Abdalla Bani and Philip Payne and Aristeidis Sotiras},
  title = {{Multimodal normative modeling in Alzheimer’s Disease with introspective variational autoencoders}},
  year = {2024},
  doi = {10.1101/2024.12.12.628273},
  howpublished = {bioRxiv},
}
Figure