Evaluating normative representation learning in generative AI for robust anomaly detection in brain imaging

Cosmin I. Bercea, Benedikt Wiestler, Daniel Rueckert, Julia A. SchnabelNature Communications 2025 · DOI

Abstract

Normative representation learning focuses on understanding the typical anatomical distributions from large datasets of medical scans from healthy individuals. Generative Artificial Intelligence (AI) leverages this attribute to synthesize images that accurately reflect these normative patterns. This capability enables the AI allowing them to effectively detect and correct anomalies in new, unseen pathological data without the need for expert labeling. Traditional anomaly detection methods often evaluate the anomaly detection performance, overlooking the crucial role of normative learning. In our analysis, we introduce novel metrics, specifically designed to evaluate this facet in AI models. We apply these metrics across various generative AI frameworks, including advanced diffusion models, and rigorously test them against complex and diverse brain pathologies. In addition, we conduct a large multi-reader study to compare these metrics to experts' evaluations. Our analysis demonstrates that models proficient in normative learning exhibit exceptional versatility, adeptly detecting a wide range of unseen medical conditions. Our code is available at https://github.com/compai-lab/2024-ncomms-bercea.git .

Introduction

The detection of brain abnormalities via AI typically relies on supervised learning, which necessitates extensive expert-labeled datasets—a significant hurdle for rare pathologies. Unsupervised Anomaly Detection (UAD) offers a paradigm shift by training models exclusively on healthy anatomy to identify deviations as "abnormal." However, a critical "evaluation gap" exists: current research primarily measures final detection accuracy (e.g., Dice scores), neglecting the model's internal ability to reconstruct a "pseudo-healthy" version of a pathological scan. If a model's representation of "normal" is flawed, its diagnostic reliability suffers. This study introduces a framework to quantify "normative representation learning" as a prerequisite for robust anomaly detection.

Methodology

The evolution of generative AI for UAD is categorized into four distinct eras: Autoencoders (AEs), Variational Autoencoders (VAEs) and GANs, Hybrid Models, and the current state-of-the-art Diffusion Models with Guided Restoration (e.g., AutoDDPM, PHANES). These models were trained on healthy brain anatomy from the FastMRI+ and IXI datasets.

Evolution of generative AI architectures and guided restoration techniques for unsupervised anomaly detection.

To evaluate these models beyond simple detection, we propose three novel metrics:

  • Restoration Quality Index (RQI): Quantifies the semantic accuracy and realism of the reconstructed healthy image.
  • Anomaly to Healthy Index (AHI): Assesses whether the distribution of reconstructed images aligns with a reference set of real healthy brains.
  • Healthy Conservation and Anomaly Correction Index (CACI): Measures the model's ability to preserve healthy tissue while selectively correcting pathological regions.

Results and Evaluation

The proposed metrics were validated against a diverse set of 171 brain pathologies in FastMRI+ and stroke lesions in ATLAS v2.0. A multi-reader study involving 16 radiologists confirmed that AI-driven metrics, particularly RQI, correlate strongly with expert human judgment.

Visualization of the proposed normative learning metrics (RQI, AHI, CACI) across different generative models.

Key findings include:

  • Superiority of Diffusion Models: AutoDDPM achieved the highest performance, successfully detecting 159 out of 171 diverse pathologies. Its reconstructions were frequently indistinguishable from real images to expert readers.
  • Normative-Robustness Correlation: There is a strong correlation between high normative learning scores (RQI, CACI) and the model's ability to generalize across unseen medical conditions.
  • Diagnostic Power: The harmonic mean of RQI, AHI, and CACI emerged as the most accurate predictor of a model's overall diagnostic efficacy.

Discussion and Conclusion

This research demonstrates that the "black box" of generative AI can be made more interpretable by evaluating its "pseudo-healthy" reconstructions. Models that best understand "normal" anatomy are inherently more robust at detecting anomalies. While current 2D diffusion models show immense promise, future work will address the transition to 3D volumes to ensure spatial consistency. Furthermore, the integration of these counterfactual images into clinical workflows requires ongoing study regarding human-AI trust and the computational overhead of diffusion-based architectures.

Summary updated

Abstract

Normative representation learning focuses on understanding the typical anatomical distributions from large datasets of medical scans from healthy individuals. Generative Artificial Intelligence (AI) leverages this attribute to synthesize images that accurately reflect these normative patterns. This capability enables the AI allowing them to effectively detect and correct anomalies in new, unseen pathological data without the need for expert labeling. Traditional anomaly detection methods often evaluate the anomaly detection performance, overlooking the crucial role of normative learning. In our analysis, we introduce novel metrics, specifically designed to evaluate this facet in AI models. We apply these metrics across various generative AI frameworks, including advanced diffusion models, and rigorously test them against complex and diverse brain pathologies. In addition, we conduct a large multi-reader study to compare these metrics to experts' evaluations. Our analysis demonstrates that models proficient in normative learning exhibit exceptional versatility, adeptly detecting a wide range of unseen medical conditions. Our code is available at https://github.com/compai-lab/2024-ncomms-bercea.git .
BibTeX
@inproceedings{bercea2025evaluating,
  author = {Cosmin I. Bercea and Benedikt Wiestler and Daniel Rueckert and Julia A. Schnabel},
  title = {{Evaluating normative representation learning in generative AI for robust anomaly detection in brain imaging}},
  booktitle = {Nature Communications},
  year = {2025},
  doi = {10.1038/s41467-025-56321-y},
}
Figure