Detecting and Monitoring Bias for Subgroups in Breast Cancer Detection AI

Amit Kumar Kundu, Florence X. Doo, Vaishnavi Patil, Amitabh Varshney, Joseph F. JáJáarXiv 2025 · DOI · OpenReview

TL;DR

This paper tackles the critical issue of bias in AI models for breast cancer detection. By analyzing high-performing models on two large datasets, they identify specific subgroups that are disproportionately affected by performance degradation. To address this, they propose and validate a CUSUM-based monitoring system that can detect and alert on performance drifts in these subgroups, ensuring more equitable AI deployment.

What Problem Does This Paper Address?

Mammography is a cornerstone of early breast cancer detection, and AI models are increasingly being used to assist radiologists. However, these AI models, often trained on specific datasets, can exhibit performance degradation and bias when deployed in real-world scenarios. This is particularly concerning for underrepresented demographic and clinical subgroups, who may already face healthcare disparities. The challenge is to not only identify these biases but also to proactively monitor AI performance over time to ensure equitable outcomes for all patients. Existing methods often focus on pre-deployment bias detection but lack robust mechanisms for continuous monitoring after deployment.

Methods

The authors employ a multi-faceted approach:

  • Model Training: They train a new AI model on the large EMBED dataset using advanced computer vision techniques, including strong data augmentations, cosine annealing, label smoothing, and exponential moving averaging of model weights.
  • Subgroup Analysis: They rigorously evaluate the performance of both their trained EMBED model and a top-performing RSNA model across various subgroups. These subgroups are defined by attributes such as age, race, breast tissue density, mammogram view position, scanner type, and clinical site.
  • Performance Metrics: A range of classification metrics are used, including Positive Predictive Value (PPV), sensitivity, F1-score, accuracy, specificity, and Area Under the Receiver Operating Characteristic Curve (AUROC).
  • Statistical Significance Testing: Mann-Whitney U tests and Kruskal-Wallis tests are used to determine the statistical significance of performance differences between subgroups.
  • Performance Monitoring: They adopt Statistical Process Control (SPC) methods, specifically Cumulative Sum (CUSUM) control charts, to monitor for performance drifts over time. This method is designed to detect when a model's performance degrades for specific subgroups, triggering an alert.

Datasets

The study utilizes two large and diverse mammography datasets:

  • RSNA Screening Mammography Breast Cancer Detection dataset (Carr et al., 2022): This dataset contains 54,706 screening mammograms. While it includes attributes like tissue density, view position, and age, it notably lacks racial or ethnic information.
  • Emory Breast Imaging Dataset (EMBED) (Jeong et al., 2023): This dataset is described as large, granular, and covering racially diverse demographics. It includes screening and diagnostic mammograms along with pathological reports and relevant attributes. The authors filtered this dataset to include only MLO and CC view positions, resulting in 217,672 mammograms from 20,128 participants.

Key Insights and Ideas

  • Bias is Pervasive, Even in High-Performing Models: Even AI models that achieve high overall performance can exhibit significant performance disparities across different subgroups. This highlights that simply optimizing for aggregate metrics is insufficient for ensuring fairness.
  • Subgroup Performance Matters: The paper demonstrates that certain subgroups, such as older age groups, specific racial/ethnic groups (e.g., African American or Black), and those with denser breast tissue, can experience notably lower performance (e.g., reduced sensitivity or PPV).
  • Data Imbalance and Representation are Key: The study points to how imbalanced datasets and underrepresentation of certain groups can directly lead to biased model performance. The lack of racial information in the RSNA dataset is a prime example of a limitation that can mask biases.
  • Continuous Monitoring is Crucial: Pre-deployment bias assessment is a good first step, but the real challenge lies in maintaining performance over time as data distributions shift. The CUSUM-based monitoring system offers a proactive solution for this.
  • CUSUM for Real-Time Drift Detection: The CUSUM charts effectively flagged performance drops in simulated underperforming subgroups, demonstrating their utility in identifying degradation that might otherwise go unnoticed.

Key Results

  • EMBED Model Performance: The authors' trained EMBED-AI model achieved an AUROC of 0.9448.
  • RSNA Model Performance: The model by Nguyen (2023) achieved an AUROC of 91.6% on the RSNA dataset.
  • Identified Underperforming Subgroups:
    • For EMBED: Lower PPV was observed for the '70 ≤ age < 80' group and the African American (AAB) group. Lower sensitivity was noted for Caucasian individuals aged ≥ 80, AAB with tissue density 1, NHPI, and MLO view groups.
  • For RSNA: Lower PPV was seen for the '40 ≤ age < 50' group, and lower sensitivity for the density of 4 group.
  • CUSUM Monitoring Effectiveness: The CUSUM monitoring system demonstrated a low False Alarm Rate (FAR) and minimal average detection delay in simulations, effectively identifying performance drifts in underperforming subgroups.

Limitations and Future Work

  • Data Limitations: The absence of racial information in the RSNA dataset limits the scope of bias analysis for this dataset. For some subgroups with very few samples (e.g., AIAN, NHPI), sensitivity estimates may be unreliable, highlighting the need for more data.
  • Generalizability: While the models perform well overall, the identified performance drops in specific subgroups raise questions about their generalizability to diverse real-world populations.
  • Future Work: The authors suggest that future work could explore specialized or alternative imaging strategies for age groups that show performance decline (e.g., age ≥ 80). They also emphasize the importance of continuous monitoring to ensure models remain accurate and equitable as demographic and clinical landscapes evolve. The proposed framework can inform the development of more robust evaluation techniques for unbiased breast cancer detection models.

Summary updated

Abstract

Automated mammography screening plays an important role in early breast cancer detection. However, current machine learning models, developed on some training datasets, may exhibit performance degradation and bias when deployed in real-world settings. In this paper, we analyze the performance of high-performing AI models on two mammography datasets-the Emory Breast Imaging Dataset (EMBED) and the RSNA 2022 challenge dataset. Specifically, we evaluate how these models perform across different subgroups, defined by six attributes, to detect potential biases using a range of classification metrics. Our analysis identifies certain subgroups that demonstrate notable underperformance, highlighting the need for ongoing monitoring of these subgroups' performance. To address this, we adopt a monitoring method designed to detect performance drifts over time. Upon identifying a drift, this method issues an alert, which can enable timely interventions. This approach not only provides a tool for tracking the performance but also helps ensure that AI models continue to perform effectively across diverse populations.
BibTeX
@misc{kundu2025detecting,
  author = {Amit Kumar Kundu and Florence X. Doo and Vaishnavi Patil and Amitabh Varshney and Joseph F. JáJá},
  title = {{Detecting and Monitoring Bias for Subgroups in Breast Cancer Detection AI}},
  year = {2025},
  eprint = {2502.10562},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2502.10562},
}
Figure