Detecting Unforeseen Data Properties with Diffusion Autoencoder Embeddings Using Spine MRI Data

Robert Graf, Florian Hunecke, Soeren Pohl, Matan Atad, Hendrik Kristian Möller, Sophie Starck, Thomas Kröncke, Stefanie Bette, Fabian Bamberg, Tobias Pischon, Thoralf Niendorf, Carsten Oliver Schmidt, Johannes C. Paetzold, Daniel Rueckert, Jan S. Kirschke

Introduction

Deep learning models in medical imaging rely heavily on large-scale datasets to achieve high performance. However, these datasets often harbor hidden biases and protocol variations that can compromise model generalization and fairness. In a study presented at the MICCAI 2024 workshop on Interpretability of Machine Intelligence in Medical Image Computing (iMIMIC), researchers investigated the use of Diffusion Autoencoder (DAE) embeddings to audit over 11,000 spine MRIs from the German National Cohort (NAKO). This approach aims to uncover "unforeseen" data properties—such as subtle anatomical variations and site-specific scanner fingerprints—that are often invisible to manual inspection.

The Challenge of Large-Scale Medical Data

Large-scale studies like NAKO collect data across multiple centers over extended periods. This introduces two primary challenges for machine learning development:

  1. Hidden Biases: Models may inadvertently learn to rely on "protected variables" (e.g., sex or age) even when these features are not the intended targets, leading to biased outcomes.
  2. Protocol Variations: Subtle differences in patient positioning or scanner hardware between clinical sites can create "shortcuts." A neural network might learn to identify the acquisition site rather than the pathology, resulting in models that fail to generalize when deployed at new hospitals.

Because manually auditing tens of thousands of images for these nuances is impossible, the authors propose using unsupervised embeddings—mathematical "fingerprints" of images—to visualize and detect these properties automatically.

Methodology

The core of the study is the Diffusion Autoencoder (DAE). Unlike standard Variational Autoencoders (VAEs) or Generative Adversarial Networks (GANs), DAEs combine the high-quality reconstruction capabilities of diffusion models with a searchable, organized latent space (the embedding).

The researchers' workflow involved:

  • Training a DAE on sagittal T2-weighted spine MRIs.
  • Extracting latent embeddings for each image in the dataset.
  • Using t-SNE (a dimensionality reduction technique) to visualize how the data clusters.
  • Training Support Vector Machines (SVMs) on these embeddings to determine how well they capture specific traits like sex, age, height, and weight.
  • Applying GradCAM and image averaging to interpret the specific image features causing certain clusters to form.

Dataset and Experimental Setup

The study utilized a massive dataset from the German National Cohort (NAKO), encompassing 11,186 participants. The images consisted of sagittal T2-weighted MRIs covering three distinct spinal regions: the cervical (neck), thoracic (chest), and lumbar (lower back). Metadata included self-reported sex, age, height, weight, and the specific acquisition location (city).

Key Insights

Identifying Anatomical Variants and Bias

One of the most significant findings was the identification of a subpopulation whose embeddings placed them in the "opposite" sex cluster. Upon investigation, the researchers found that while their labels were correct, their spinal anatomy shared features typically associated with the other sex. Supervised models struggled significantly to classify these individuals, suggesting that these "normal variants" are a major source of error and potential fairness gaps in medical AI.

Site-Specific Scanner Fingerprints

The embeddings successfully clustered images by the city where they were acquired. By averaging images within these clusters, the authors discovered that different centers had consistent differences in "framing"—specifically, how the head was positioned or the degree of neck curvature caused by different MR headrests. These site-specific fingerprints represent potential shortcuts that could lead to model overfitting.

Comparative Performance

The DAE embeddings demonstrated superior "disentanglement" of features compared to StyleGAN or $\beta$-VAE. While StyleGAN could identify broad body regions, it struggled to separate sex and age as cleanly as the DAE approach. Quantitatively, an SVM using only unsupervised DAE embeddings could identify the body region with 100% accuracy and the patient's sex with 98.8% accuracy. For age prediction, the DAE achieved a Mean Absolute Error (MAE) of 3.84 years, significantly outperforming StyleGAN (5.62 years) and $\beta$-VAE (7.80 years).

Discussion and Future Work

While DAE embeddings offer a powerful tool for auditing datasets, the authors note several areas for further consideration:

  • Privacy Concerns: The ability to accurately identify an acquisition site from an embedding raises potential re-identification risks in supposedly anonymized datasets.
  • Biological Roots: The exact biological reasons for the "sex bias" group (e.g., hormonal differences or growth patterns) remain a hypothesis requiring clinical validation.
  • Generalization: Future work is needed to determine if DAE embeddings are equally perceptive across other modalities, such as CT or ultrasound.

By providing a method to automatically detect unforeseen data properties, this research moves toward more transparent and robust medical AI systems.

Summary updated

BibTeX
@inproceedings{graf2024detecting,
  author = {Robert Graf and Florian Hunecke and Soeren Pohl and Matan Atad and Hendrik Kristian Möller and Sophie Starck and Thomas Kröncke and Stefanie Bette and Fabian Bamberg and Tobias Pischon and Thoralf Niendorf and Carsten Oliver Schmidt and Johannes C. Paetzold and Daniel Rueckert and Jan S. Kirschke},
  title = {{Detecting Unforeseen Data Properties with Diffusion Autoencoder Embeddings Using Spine MRI Data}},
  booktitle = {ISIC/iMIMIC/EARTH/DeCaF @ MICCAI},
  year = {2024},
  doi = {10.1007/978-3-031-77610-6\_8},
}
Figure