Multimodal normative modeling in Alzheimers Disease with introspective variational autoencoders

Sayantan Kumar, Peijie Qiu, Aristeidis Sotiras

TL;DR

The authors introduce mmSIVAE, a multimodal soft-introspective variational autoencoder designed for normative modeling in Alzheimer’s Disease (AD). By combining an adversarial training objective with Mixture-of-Product-of-Experts (MOPOE) posterior aggregation, the model maps healthy brain distributions more accurately and identifies individual-level disease deviations across MRI and PET imaging with significantly higher sensitivity than standard VAE baselines.

What Problem Does This Paper Address?

Alzheimer’s Disease is notoriously heterogeneous; its clinical presentation, progression rate, and underlying pathology vary drastically from person to person. Traditional "case-control" studies look at group-level averages, which can wash out these crucial individual differences.

To capture this heterogeneity, researchers use normative modeling—think of it like a pediatric growth chart, but for the brain. The model learns the "normal" distribution of brain features from a healthy population and quantifies how much an individual patient deviates from that norm.

While Variational Autoencoders (VAEs) are popular tools for this task, they suffer from two major flaws when applied to multimodal neuroimaging data:

  1. Imperfect Reference Fitting: Standard VAEs often fail to map the healthy reference distribution tightly enough. This leads to large reconstruction errors even for healthy controls, inflating false-positive rates.
  2. Brittle Multimodal Fusion: Combining different imaging modalities (like structural MRI and molecular PET scans) in a shared latent space is difficult. Common aggregation methods either let a single precise modality dominate the latent space (Product-of-Experts) or blur the distributions together (Mixture-of-Experts), resulting in an uninformative joint representation.

Methods

The proposed framework, mmSIVAE (multimodal soft-introspective VAE), tackles both limitations simultaneously:

  • Soft-Introspective Training: Instead of using a standard VAE training objective, mmSIVAE adopts an adversarial approach. The encoder is trained to maximize the Evidence Lower Bound (ELBO) for real healthy data while minimizing it for generated ("fake") data. This forces the encoder to act as a discriminator, pushing generated samples away from the true healthy latent manifold and sharpening the model's understanding of "normal."
  • MOPOE Aggregation: To fuse modalities effectively, the authors use Mixture-of-Product-of-Experts (MOPOE). This technique computes Product-of-Experts (PoE) posteriors for all possible subsets of modalities and then combines them using a Mixture-of-Experts (MoE) formulation. This achieves a sharp, informative joint latent space without allowing one modality to completely suppress another.
Training flow of the mmSIVAE framework, illustrating the min-max game between encoders and decoders.

The authors calculate patient-specific deviation scores using the Mahalanobis distance in both the latent space ($D_{ml}$) and the feature reconstruction space ($D_{mf}$).

Datasets

The model was trained and evaluated using data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI):

  • Training Set: 248 cognitively unimpaired, amyloid-negative healthy controls.
  • Evaluation Set: A separate holdout cohort of 48 healthy controls and a disease cohort of 726 amyloid-positive individuals spanning the AD spectrum (preclinical, mild cognitive impairment, and severe dementia).
  • Features: 90 regional brain metrics (66 cortical, 24 subcortical) consisting of regional volumes from T1-weighted MRI and Standardized Uptake Value Ratios (SUVR) from AV45 Amyloid PET.

Key Insights and Ideas

  • Adversarial Pressure Sharpens the Manifold: Standard VAEs suffer from a "reconstruction blur" because they aren't explicitly penalized for finding trivial, overly generalized solutions. Introspective training forces the model to draw a strict boundary around what constitutes a "healthy control," which directly translates to better out-of-distribution (OOD) anomaly detection.
  • Latent Space is Better for Outlier Detection: Computing deviation scores within the fused latent space ($D_{ml}$) consistently yielded higher diagnostic sensitivity than computing them in the raw feature reconstruction space ($D_{mf}$). Fusing the data early into a low-dimensional space preserves the non-linear interactions between structural atrophy (MRI) and amyloid accumulation (PET).
  • Sparsity Improves Interpretability: By identifying exactly which latent dimensions show statistically significant deviations and back-propagating only those through the decoders, the authors successfully mapped abstract latent anomalies back to specific, clinically recognizable brain regions.

Key Results

  • Superior Reconstruction: mmSIVAE achieved the lowest reconstruction error on held-out healthy controls across virtually all cortical and subcortical brain regions compared to unimodal and standard multimodal VAEs.
Average reconstruction error maps across different VAE architectures for MRI and Amyloid PET modalities.
  • Enhanced Outlier Detection: For latent space deviations ($D_{ml}$), mmSIVAE with MOPOE achieved a peak positive likelihood ratio of 10.2 (at $d=15$ latent dimensions), substantially outperforming standard multimodal VAEs (which peaked at 8.10) and unimodal baselines.
  • Clearer Cohort Separation: The feature-space deviations ($D_{mf}$) derived from mmSIVAE showed the highest Earth Mover’s Distance (34.043) between healthy controls and AD-spectrum patients, demonstrating a much cleaner separation between cohorts.
Histogram showing the distribution and separation of feature Mahalanobis deviations between training and test cohorts.
  • Clinical Alignment: The regional deviation maps successfully localized known AD hallmarks: temporal/hippocampal atrophy on MRI and frontal/precuneus amyloid accumulation on PET scans.

Limitations and Future Work

  • Gaussian Assumptions: The calculation of Mahalanobis distances assumes a Gaussian distribution of reference embeddings and reconstruction errors, which can be sensitive to covariance estimation errors in higher dimensions.
  • Domain Shifts: The model was evaluated entirely within the ADNI dataset. Performance under explicit domain shifts (e.g., different scanners, diverse clinical sites) remains an open question.
  • Missing Modalities: The current framework assumes all modalities are present during inference. Developing robust methods to handle missing clinical data streams is a critical next step for real-world deployment.

Summary updated

Abstract

Normative modeling learns a healthy reference distribution and quantifies subject-specific deviations to capture heterogeneous disease effects. In Alzheimers disease (AD), multimodal neuroimaging offers complementary signals but VAE-based normative models often (i) fit the healthy reference distribution imperfectly, inflating false positives, and (ii) use posterior aggregation (e.g., PoE/MoE) that can yield weak multimodal fusion in the shared latent space. We propose mmSIVAE, a multimodal soft-introspective variational autoencoder combined with Mixture-of-Product-of-Experts (MOPOE) aggregation to improve reference fidelity and multimodal integration. We compute deviation scores in latent space and feature space as distances from the learned healthy distributions, and map statistically significant latent deviations to regional abnormalities for interpretability. On ADNI MRI regional volumes and amyloid PET SUVR, mmSIVAE improves reconstruction on held-out controls and produces more discriminative deviation scores for outlier detection than VAE baselines, with higher likelihood ratios and clearer separation between control and AD-spectrum cohorts. Deviation maps highlight region-level patterns aligned with established AD-related changes. More broadly, our results highlight the importance of training objectives that prioritize reference-distribution fidelity and robust multimodal posterior aggregation for normative modeling, with implications for deviation-based analysis across multimodal clinical data.
BibTeX
@misc{kumar2026multimodal,
  author = {Sayantan Kumar and Peijie Qiu and Aristeidis Sotiras},
  title = {{Multimodal normative modeling in Alzheimers Disease with introspective variational autoencoders}},
  year = {2026},
  eprint = {2602.08077},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2602.08077},
}
Figure