Evaluating Normative Learning in Generative AI for Robust Medical Anomaly Detection

Cosmin I. Bercea, Benedikt Wiestler, Daniel Rueckert, Julia A. SchnabelResearch Square 2023 · DOI

Abstract

In Generative Artificial Intelligence (AI) for medical imaging, normative learning involves training AI models on large datasets of typical images from healthy volunteers, such as MRIs or CT scans. These models acquire the distribution of normal anatomical structures, allowing them to effectively detect and correct anomalies in new, unseen pathological data. This approach allows the detection of unknown pathologies without the need for expert labeling. Traditional anomaly detection methods often evaluate the anomaly detection performance, overlooking the crucial role of normative learning. In our analysis, we introduce novel metrics, specifically designed to evaluate this facet in AI models. We apply these metrics across various generative AI frameworks, including advanced diffusion models, and rigorously test them against complex and diverse brain pathologies. Our analysis demonstrates that models proficient in normative learning exhibit exceptional versatility, adeptly detecting a wide range of unseen medical conditions.

Introduction

Unsupervised Anomaly Detection (UAD) represents a significant frontier in medical imaging, offering the potential to identify pathologies without the exhaustive requirement for expert-labeled datasets. By training models exclusively on healthy data, UAD systems aim to flag any deviation from the "norm" in clinical settings. This is particularly vital for rare diseases where large-scale labeled data is unavailable.

However, current UAD research often suffers from a narrow focus on final detection performance—such as identifying a specific tumor—while neglecting the underlying mechanism: normative learning. Without a deep understanding of healthy anatomy, a model's ability to generate an "anomaly map" is fundamentally limited, often leading to biases toward high-contrast anomalies while missing subtle structural changes. This paper shifts the focus toward mastering normative learning as the key to robust anomaly detection.

Methods and Frameworks

The study presents a comprehensive evaluation of generative AI architectures, tracing their evolution from basic Autoencoders (AEs) to state-of-the-art Diffusion Models. A critical distinction is made between standard reconstruction and "Guided Restoration," where intelligent masking forces the model to rely on its learned anatomical priors to fill in missing regions.

Evolution of Generative AI architectures and guided restoration techniques.

Normative Learning Metrics

To quantify a model's understanding of healthy anatomy, the authors introduce three novel metrics:

  1. Restoration Quality Index (RQI): Evaluates the precision with which a model reproduces healthy tissue it has not previously encountered.
  2. Anomaly to Healthy Index (AHI): Measures the model's ability to "normalize" a pathology, effectively transforming abnormal structures back into plausible healthy tissue.
  3. Conservation and Anomaly Correction Index (CACI): Assesses the proficiency in preserving healthy structures while simultaneously correcting pathological areas.

Experimental Setup

The models were trained and evaluated using high-quality brain MRI datasets:

  • Training: The FastMRI+ and IXI datasets provided T1-weighted brain MRIs from healthy volunteers to establish the normative baseline.
  • Evaluation: Pathological data was sourced from FastMRI+ (643 annotated pathologies across 30 classes) and ATLAS v2.0 (specialized for stroke lesions of varying sizes and intensities).

Key Results and Insights

The analysis revealed a strong positive correlation between a model’s performance on normative metrics and its diagnostic accuracy. Models that excel at "imagining" a healthy version of a pathological scan are inherently more versatile at detecting diverse, unseen conditions.

Visualization of the three proposed Normative Learning Metrics (RQI, AHI, CACI) across different models.

Performance of Diffusion Models

Diffusion-based models, particularly autoDDPM, emerged as the new standard, significantly outperforming older GAN and VAE architectures. These models captured complex, high-frequency anatomical details without the "mode collapse" common in GANs. The autoDDPM model successfully detected 159 out of 171 unique pathologies in the FastMRI+ dataset.

The Advantage of Guided Restoration

The results highlight that "guided restoration" via masking is a superior strategy. By preventing the model from simply copying the anomaly into the output, masking forces the AI to synthesize tissue based on its internal understanding of healthy anatomy. For instance, PHANES demonstrated superior segmentation of large stroke lesions, achieving the highest Dice coefficients in the ATLAS dataset.

Identifying Pitfalls

The study also identified specific challenges, such as "paranasal sinus opacification." These areas often resulted in false positives due to their proximity to the skull, showing a rare negative correlation with normative metrics and highlighting the need for careful interpretation of UAD outputs in certain anatomical regions.

Discussion and Limitations

While normative learning significantly advances UAD, several hurdles remain. Motion artifacts in scans can mimic subtle white matter lesions (WML), leading to differentiation difficulties for both AI and human experts. Additionally, annotation bias in stroke datasets—where healthy tissue within a lesion zone is labeled as "pathology"—can penalize models that accurately preserve that healthy tissue.

Future research will focus on improving the computational efficiency of these complex models and enhancing their "explainability" to facilitate integration into fast-paced clinical workflows. By prioritizing the model's ability to learn the "norm," we can develop more robust and generalizable tools for medical anomaly detection.

Summary updated

Abstract

<title>Abstract</title> In Generative Artificial Intelligence (AI) for medical imaging, normative learning involves training AI models on large datasets of typical images from healthy volunteers, such as MRIs or CT scans. These models acquire the distribution of normal anatomical structures, allowing them to effectively detect and correct anomalies in new, unseen pathological data. This approach allows the detection of unknown pathologies without the need for expert labeling. Traditional anomaly detection methods often evaluate the anomaly detection performance, overlooking the crucial role of normative learning. In our analysis, we introduce novel metrics, specifically designed to evaluate this facet in AI models. We apply these metrics across various generative AI frameworks, including advanced diffusion models, and rigorously test them against complex and diverse brain pathologies. Our analysis demonstrates that models proficient in normative learning exhibit exceptional versatility, adeptly detecting a wide range of unseen medical conditions.
BibTeX
@inproceedings{bercea2023evaluating,
  author = {Cosmin I. Bercea and Benedikt Wiestler and Daniel Rueckert and Julia A. Schnabel},
  title = {{Evaluating Normative Learning in Generative AI for Robust Medical Anomaly Detection}},
  booktitle = {Research Square},
  year = {2023},
  doi = {10.21203/rs.3.rs-3749187/v1},
}
Figure