MIMM-X: Disentangling Spurious Correlations for Medical Image Analysis

Louisa Fay, Hajer Reguigui, Bin Yang, Sergios Gatidis, Thomas KüstnerLecture Notes in Computer Science 2025 · DOI · arXiv · OpenAlex

Abstract

Deep learning models in medical imaging often rely on "shortcuts"—spurious correlations such as patient demographics or scanner-specific artifacts—rather than biological markers of disease. This paper introduces MIMM-X (Mutual Information Minimization Model - X), a framework designed for multi-factor disentanglement. By partitioning latent features and minimizing mutual information between primary and confounding variables, MIMM-X ensures that models remain robust and fair, even when deployed in clinical environments where training-set biases are inverted.

Introduction: The Shortcut Problem in Medical AI

Deep learning models are notoriously efficient at finding the path of least resistance. In medical imaging, if a training dataset contains a superficial pattern that correlates with a target label, the model will often learn that shortcut instead of the complex biological features it is intended to identify. For instance, if a specific disease is more prevalent in one sex within a training set, a model might use "sex" as a proxy for the disease.

While previous methods could handle a single known shortcut, real-world medical data is a complex web of multiple, entangled factors: different scanner types, varying acquisition protocols, and diverse patient demographics. MIMM-X addresses this challenge of multi-factor disentanglement, forcing models to ignore several different spurious correlations simultaneously without compromising their ability to learn the primary task.

Methodology: The MIMM-X Framework

Causal graph and MIMM-X architecture showing feature partitioning and mutual information minimization.

The authors propose three key architectural innovations to achieve robust disentanglement:

  1. Feature Partitioning: The feature encoder splits the latent representation into $N+1$ subvectors. One subvector ($f_y$) is dedicated to the primary task (e.g., disease detection), while the others ($f_Z$) represent the $N$ different spurious factors the model should ignore.
  2. Mutual Information Neural Estimation (MINE): To ensure the primary features ($f_y$) contain zero information about the spurious factors ($f_Z$), the model minimizes the Mutual Information (MI) between them. MIMM-X scales this efficiently by stacking all spurious factors and estimating a single MI value.
  3. Confounder Attention Weighter (CAW) & Dynamic Loss Scaling (DLS): Learning to ignore shortcuts is often more difficult than learning the shortcuts themselves. MIMM-X uses CAW to adaptively weight which spurious factors require more suppression and DLS to prevent the primary task signal from being overwhelmed by penalty terms during training.

Experimental Setup and Datasets

Overview of the three large-scale experiments and the confounding factors addressed.

The framework was validated across three large-scale experiments involving two imaging modalities:

  • Brain MRI: Using data from the German National Cohort (NAKO) and the UK Biobank (UKB) for age prediction, with sex and dataset origin as spurious factors.
  • Chest X-ray: Using the CheXpert dataset for pleural effusion detection, with sex, age, and co-occurring diseases (CoD) as confounding variables.

To prove the models were not merely memorizing dataset-specific biases, they were tested on Inverted sets (where training correlations are flipped) and Balanced sets.

Key Results and Discussion

Robustness Against Bias

Standard baseline models performed well on validation data but collapsed when tested on inverted data. In the Brain MRI task, the baseline's accuracy plummeted from 96% to 27.2% when correlations were reversed. In contrast, MIMM-X maintained a high 82.8% accuracy, demonstrating significant robustness. In Chest X-ray experiments, MIMM-X achieved the highest accuracy on inverted test sets (70.4%), while other methods hovered near random guessing (40-50%).

Measurable Disentanglement

The authors utilized "cross-prediction" as a litmus test: if the primary feature subvector ($f_y$) can still predict a patient's sex or age, disentanglement has failed. MIMM-X achieved near "random guess" performance (50%) on these cross-tasks, proving the information was effectively scrubbed. Visual analysis via t-SNE plots further confirmed this; while baseline features clustered by demographics, MIMM-X features appeared as a "mixed soup," indicating the model had successfully stopped using those categories to organize its knowledge.

Conclusion and Future Work

MIMM-X represents a significant step toward reliable medical AI by addressing the multi-factor nature of clinical bias. Currently, the framework requires researchers to specify confounding factors in advance. Future research will focus on automatically discovering unknown spurious correlations and scaling the model to handle even more complex, non-linear confounding structures in multi-center clinical trials.

Summary updated

BibTeX
@inproceedings{fay2025mimm,
  author = {Louisa Fay and Hajer Reguigui and Bin Yang and Sergios Gatidis and Thomas Küstner},
  title = {{MIMM-X: Disentangling Spurious Correlations for Medical Image Analysis}},
  booktitle = {Lecture Notes in Computer Science},
  year = {2025},
  doi = {10.1007/978-3-032-05870-6\_10},
}
Figure