A Study of Age and Sex Bias in Multiple Instance Learning based Classification of Acute Myeloid Leukemia Subtypes

Ario Sadafi, Matthias Hehr, Nassir Navab, Carsten Marr

TL;DR

This study reveals significant age and sex biases in Multiple Instance Learning (MIL) models used to classify Acute Myeloid Leukemia (AML) subtypes. The researchers demonstrate that models trained on imbalanced datasets perform poorly on underrepresented groups—particularly females and the elderly—highlighting a critical need for inclusive data to ensure equitable AI-driven clinical diagnostics.

What Problem Does This Paper Address?

Acute Myeloid Leukemia (AML) is a fast-acting blood cancer where rapid, accurate subtype identification is a matter of life and death. While deep learning, specifically Multiple Instance Learning (MIL), has shown great promise in automating this classification from blood smear images, these models are only as good as the data they learn from.

The medical community has long grappled with bias; for decades, heart attack symptoms in women were misunderstood because research focused primarily on men. The authors of this paper ask: Are we repeating these mistakes in AI? Despite the rise of AI in hematology, there is a startling lack of research into how demographic imbalances in training data—specifically age and sex—affect the fairness and accuracy of leukemia diagnosis.

Methods

The researchers employed a two-stage computer vision pipeline:

  1. Feature Extraction: A ResNet34 model, pre-trained on over 300,000 single-cell images, was used to convert individual cell images into high-dimensional feature vectors.
  2. Multiple Instance Learning (MIL): Because a single patient's diagnosis is based on a "bag" of many cells, the authors used an attention-based MIL architecture. This allows the model to weigh the importance of specific cells within a patient's sample to reach a final subtype classification.

To test for sex bias, they trained five different models with varying proportions of female patients (0%, 25%, 50%, 75%, and 100%) and evaluated them on sex-specific holdout sets. For age bias, they used a "leave-one-group-out" approach, training on four age brackets and testing on the fifth (e.g., training on everyone except the elderly to see how the model fails on that specific demographic).

Datasets

The study utilized a curated dataset from the Munich Leukemia Laboratory (MLL), consisting of:

  • 189 individuals (after rigorous data cleaning).
  • 81,214 single-cell images.
  • Four AML genetic subtypes: PML::RARA, NPM1 mutation, CBFB::MYH11 fusion, and RUNX1::RUNX1T1 fusion, plus a healthy control group.
  • Demographics: The cohort included 77 females and 112 males, with ages ranging from 19 to 86 years.
Fig. 1. Dataset composition. A) Distribution of sex in the dataset. B) AML genetic subtype distribution among male and female individuals. C) Overal distribution of AML genetic subtypes among the individuals in the dataset. D) Age distribution across five age groups, alongside the genetic subtype distribution within each age group.

Key Insights and Ideas

  • The "Balanced is Best" Rule: For the RUNX1::RUNX1T1 subtype, the highest accuracy was achieved only when the training set had a balanced 50/50 sex ratio. Deviating from this balance caused performance to dip on both male and female test sets.
  • Female Vulnerability to Imbalance: The study found that female patients were more significantly affected by sex-imbalanced training data than males. When females were underrepresented in training, the model's ability to diagnose them correctly dropped sharply.
  • The Elderly Diagnostic Gap: One of the most striking findings was the model's failure to identify the RUNX1::RUNX1T1 subtype in elderly patients (ages 72-86) when they were excluded from training. The authors note that sample quality is often worse in elderly patients and this specific subtype is less common in that age group, creating a "perfect storm" for AI bias.
  • Bias is Upstream: The authors argue that data bias doesn't start at image acquisition; it starts at patient selection and study design. If clinical trials or hospital archives are demographically skewed, the resulting AI will be too.

Key Results

The researchers used the Area Under the Precision-Recall Curve (AUPRC) to measure performance:

  • Sex Bias: In the female-only test set, performance for the PML::RARA subtype improved significantly as the proportion of females in the training set increased. Conversely, models trained only on males performed poorly on females for several subtypes.
Fig. 2. Area under precision-recall (PR) curve for each class, representing models trained with varying proportions of females versus males in the training set. The mod- els were tested on two separate test sets: A) the female-only test set, and B) the male-only test set. Statistical significance was determined using The statistical anal- ysis involved the Kruskal-Wallis test followed by post hoc pairwise Dunn test with Bonferroni correction, indicated by asterisks: * (0.1 ≥p > 0.05), ** (0.05 ≥p > 0.01), *** (0.01 ≥p > 0.001)
  • Age Bias: The model's ability to detect the RUNX1::RUNX1T1 subtype in the elderly was significantly lower compared to younger groups when that age bracket was underrepresented. Similarly, excluding young adults from training compromised the classification of NPM1 and CBFB::MYH11 subtypes.
Fig. 3. Area under precision-recall (PR) curve for every class is reported for models trained with different age groups represented in the training set. The test set age group is always held out. Statistical significance was determined using Kruskal-Wallis test followed by post hoc pairwise Dunn test with Bonferroni correction, indicated by asterisks: * (0.1 ≥p > 0.05), ** (0.05 ≥p > 0.01), *** (0.01 ≥p > 0.001)

Limitations and Future Work

The primary limitation is the sample size; 189 individuals is relatively small for deep learning, which may exacerbate the observed biases. The authors also noted that the control group lacked elderly individuals, preventing a full bias analysis for healthy samples.

Moving forward, the researchers call for:

  • Multicentric Datasets: Collecting larger, more diverse datasets from multiple institutions to improve generalizability.
  • Fairness-Aware Learning: Integrating algorithmic constraints during training that explicitly penalize biased outcomes.
  • Methodological Oaths: Suggesting that machine learning scientists should adopt a version of the physician's oath to prioritize fair and just medical care in their algorithmic designs.

Summary updated

Abstract

Accurate classification of Acute Myeloid Leukemia (AML) subtypes is crucial for clinical decision-making and patient care. In this study, we investigate the potential presence of age and sex bias in AML subtype classification using Multiple Instance Learning (MIL) architectures. To that end, we train multiple MIL models using different levels of sex imbalance in the training set and excluding certain age groups. To assess the sex bias, we evaluate the performance of the models on male and female test sets. For age bias, models are tested against underrepresented age groups in the training data. We find a significant effect of sex and age bias on the performance of the model for AML subtype classification. Specifically, we observe that females are more likely to be affected by sex imbalance dataset and certain age groups, such as patients with 72 to 86 years of age with the RUNX1::RUNX1T1 genetic subtype, are significantly affected by an age bias present in the training data. Ensuring inclusivity in the training data is thus essential for generating reliable and equitable outcomes in AML genetic subtype classification, ultimately benefiting diverse patient populations.
BibTeX
@inproceedings{sadafi2023study,
  author = {Ario Sadafi and Matthias Hehr and Nassir Navab and Carsten Marr},
  title = {{A Study of Age and Sex Bias in Multiple Instance Learning based Classification of Acute Myeloid Leukemia Subtypes}},
  booktitle = {CLIP/FAIMI/EPIMI @ MICCAI},
  year = {2023},
  doi = {10.1007/978-3-031-45249-9\_25},
}
Figure