AI Alignment in Medical Imaging: Unveiling Hidden Biases Through Counterfactual Analysis

Haroui Ma, Francesco Quinzan, Theresa Willem, Stefan BauerarXiv 2025 · DOI · arXiv

TL;DR

This paper introduces a novel statistical framework to detect and quantify biases in AI models used for medical imaging. By leveraging counterfactual invariance, the method assesses how model predictions change when sensitive attributes (like race) are hypothetically altered, offering a more robust fairness evaluation than traditional association-based metrics.

What Problem Does This Paper Address?

Medical AI systems, particularly those for image analysis, have shown impressive diagnostic capabilities. However, a critical concern is their susceptibility to biases, which can lead to disparities in performance across different demographic groups. These biases can arise from confounding factors in the training data, where AI models might inadvertently learn to associate sensitive attributes (e.g., race, age, sex) with specific medical conditions, even without a causal link. This can result in unfairness, underdiagnosis, and undertreatment for certain patient populations. Existing fairness metrics often rely on statistical associations, which may not capture the true causal impact of sensitive attributes on model predictions. This paper aims to address this gap by proposing a method that directly tests for counterfactual invariance, ensuring that a model's predictions remain stable when sensitive attributes are changed, independent of their causal relationship to the disease.

Methods

The core of this paper is a novel statistical test for Counterfactual Invariance (CI). The method is built upon the idea that a fair AI model should produce the same prediction regardless of changes to sensitive attributes, assuming all other relevant factors remain constant.

Key methodological contributions include:

  • Counterfactual Invariance Test (CIT-LR): This statistical test is designed to assess CI from samples. It leverages a theoretical framework that connects CI to quantities directly estimable from data.
  • Conditional Latent Diffusion Models (CLDMs): To generate realistic counterfactual images (i.e., images where a sensitive attribute is altered while preserving disease-related features), the authors employ CLDMs. These models are trained to generate high-fidelity images conditioned on specific attributes.
  • Disentangled Representations: To improve the quality and control of counterfactual image generation, the CLDMs are enhanced with disentangled representations. This aims to separate the representation of sensitive attributes from disease-related features.
  • Algorithm 1 (CIT-LR): This algorithm outlines the practical steps for implementing the CI test. It involves training a CLDM with disentangled representations to generate counterfactual data, learning functions that estimate the expected outcome under different attribute conditions, and finally, using a two-sample Student's t-test to compare these expectations.

The approach is novel in its use of diffusion models for generating counterfactual medical images and its direct application of counterfactual invariance principles to test pre-trained diagnostic models, rather than solely focusing on fairness during training.

Datasets

The authors utilize a combination of synthetic and real-world medical imaging datasets for their experiments:

  • Synthetic Datasets: Six synthetic datasets are created, named "linear," "quadratic," "exponential," "interactive," "log-exponent," and "sin." These datasets are designed with known data-generating processes (DGPs) to allow for controlled evaluation of the proposed method.
  • CHEXPERT Dataset: A large dataset of chest radiographs with associated labels for common radiographic observations (e.g., cardiomegaly, edema) and demographic attributes (race).
  • MIMIC-CXR Dataset: Another large, publicly available dataset of chest X-rays with radiological reports and demographic information, similar in scope to CHEXPERT.

The authors filter these real-world datasets to focus on specific diseases and racial groups ("black," "white," and "asian") for consistency in their experiments.

Key Insights and Ideas

  • Counterfactual Invariance is Crucial for Medical AI Fairness: The paper strongly argues that traditional association-based fairness metrics are insufficient for medical AI. True fairness requires that predictions are invariant to changes in sensitive attributes, even if those attributes are not causally linked to the disease.
  • Diffusion Models are Powerful for Counterfactual Generation: The successful use of CLDMs with disentangled representations demonstrates their capability in generating realistic medical images with specific attribute manipulations, a key enabler for counterfactual analysis.
  • Disentanglement Improves Generation Quality: Incorporating disentangled representations in the diffusion models leads to better generation performance, which is crucial for obtaining reliable counterfactual data.
  • CIT-LR Aligns Better with True Invariance: The proposed CIT-LR method shows a stronger correlation with the ground truth of counterfactual invariance (measured by ECA) compared to standard metrics like Demographic Parity (DP) and Equality of Opportunity (EO). This suggests it's a more sensitive detector of bias.

Key Results

  • Alignment with Counterfactual Accuracy (ECA): The proposed CIT-LR test demonstrates a significantly higher correlation with the Expected Counterfactual Accuracy (ECA) metric across synthetic and real-world datasets compared to DP and EO. This indicates that CIT-LR is more effective at identifying models that are truly counterfactually invariant.
  • Performance on Real-World Datasets: Experiments on CHEXPERT and MIMIC-CXR show that CIT-LR correctly flags models lacking CI (yielding significant p-values) and correctly identifies models with high CI (yielding non-significant p-values), outperforming DP and EO in consistency.
  • Generative Model Performance: The Disentangled Latent Diffusion Model (DLDM) shows improved performance in terms of counterfactual effectiveness and FID scores compared to baselines, highlighting the quality of generated counterfactual images.
  • Disease-Specific Biases: The study reveals that biases can be disease-specific. For instance, models might exhibit CI for one disease but not another, underscoring the need for granular fairness evaluation.

Limitations and Future Work

  • Hard-to-Control Variance and Complexity of Diffusion Models: The authors acknowledge that training diffusion models can be challenging due to their inherent variance and complexity.
  • Assumption of Latent Representation Completeness: The method relies on the assumption that latent representations capture all relevant factors. This assumption is not rigorously proven and could be a limitation if crucial information is missed.
  • Definition of "Protected" Attributes: The paper notes that defining what constitutes a "protected" attribute requires careful consideration and consultation with medical professionals, as these are often social and political constructs.
  • Future Work: The authors suggest refining representations, addressing unobserved confounding, and expanding the methodology to a broader range of problems. They emphasize the importance of interdisciplinary collaboration for advancing responsible AI in healthcare.

Summary updated

Abstract

Machine learning (ML) systems for medical imaging have demonstrated remarkable diagnostic capabilities, but their susceptibility to biases poses significant risks, since biases may negatively impact generalization performance. In this paper, we introduce a novel statistical framework to evaluate the dependency of medical imaging ML models on sensitive attributes, such as demographics. Our method leverages the concept of counterfactual invariance, measuring the extent to which a model's predictions remain unchanged under hypothetical changes to sensitive attributes. We present a practical algorithm that combines conditional latent diffusion models with statistical hypothesis testing to identify and quantify such biases without requiring direct access to counterfactual data. Through experiments on synthetic datasets and large-scale real-world medical imaging datasets, including \textsc{cheXpert} and MIMIC-CXR, we demonstrate that our approach aligns closely with counterfactual fairness principles and outperforms standard baselines. This work provides a robust tool to ensure that ML diagnostic systems generalize well, e.g., across demographic groups, offering a critical step towards AI safety in healthcare. Code: https://github.com/Neferpitou3871/AI-Alignment-Medical-Imaging.
BibTeX
@misc{ma2025ai,
  author = {Haroui Ma and Francesco Quinzan and Theresa Willem and Stefan Bauer},
  title = {{AI Alignment in Medical Imaging: Unveiling Hidden Biases Through Counterfactual Analysis}},
  year = {2025},
  eprint = {2504.19621},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2504.19621},
}
Figure