TL;DR
This study reveals significant age and sex biases in Multiple Instance Learning (MIL) models used to classify Acute Myeloid Leukemia (AML) subtypes. The researchers demonstrate that models trained on imbalanced datasets perform poorly on underrepresented groups—particularly females and the elderly—highlighting a critical need for inclusive data to ensure equitable AI-driven clinical diagnostics.
What Problem Does This Paper Address?
Acute Myeloid Leukemia (AML) is a fast-acting blood cancer where rapid, accurate subtype identification is a matter of life and death. While deep learning, specifically Multiple Instance Learning (MIL), has shown great promise in automating this classification from blood smear images, these models are only as good as the data they learn from.
The medical community has long grappled with bias; for decades, heart attack symptoms in women were misunderstood because research focused primarily on men. The authors of this paper ask: Are we repeating these mistakes in AI? Despite the rise of AI in hematology, there is a startling lack of research into how demographic imbalances in training data—specifically age and sex—affect the fairness and accuracy of leukemia diagnosis.
Methods
The researchers employed a two-stage computer vision pipeline:
- Feature Extraction: A ResNet34 model, pre-trained on over 300,000 single-cell images, was used to convert individual cell images into high-dimensional feature vectors.
- Multiple Instance Learning (MIL): Because a single patient's diagnosis is based on a "bag" of many cells, the authors used an attention-based MIL architecture. This allows the model to weigh the importance of specific cells within a patient's sample to reach a final subtype classification.
To test for sex bias, they trained five different models with varying proportions of female patients (0%, 25%, 50%, 75%, and 100%) and evaluated them on sex-specific holdout sets. For age bias, they used a "leave-one-group-out" approach, training on four age brackets and testing on the fifth (e.g., training on everyone except the elderly to see how the model fails on that specific demographic).
Datasets
The study utilized a curated dataset from the Munich Leukemia Laboratory (MLL), consisting of:
- 189 individuals (after rigorous data cleaning).
- 81,214 single-cell images.
- Four AML genetic subtypes: PML::RARA, NPM1 mutation, CBFB::MYH11 fusion, and RUNX1::RUNX1T1 fusion, plus a healthy control group.
- Demographics: The cohort included 77 females and 112 males, with ages ranging from 19 to 86 years.
Key Insights and Ideas
- The "Balanced is Best" Rule: For the RUNX1::RUNX1T1 subtype, the highest accuracy was achieved only when the training set had a balanced 50/50 sex ratio. Deviating from this balance caused performance to dip on both male and female test sets.
- Female Vulnerability to Imbalance: The study found that female patients were more significantly affected by sex-imbalanced training data than males. When females were underrepresented in training, the model's ability to diagnose them correctly dropped sharply.
- The Elderly Diagnostic Gap: One of the most striking findings was the model's failure to identify the RUNX1::RUNX1T1 subtype in elderly patients (ages 72-86) when they were excluded from training. The authors note that sample quality is often worse in elderly patients and this specific subtype is less common in that age group, creating a "perfect storm" for AI bias.
- Bias is Upstream: The authors argue that data bias doesn't start at image acquisition; it starts at patient selection and study design. If clinical trials or hospital archives are demographically skewed, the resulting AI will be too.
Key Results
The researchers used the Area Under the Precision-Recall Curve (AUPRC) to measure performance:
- Sex Bias: In the female-only test set, performance for the PML::RARA subtype improved significantly as the proportion of females in the training set increased. Conversely, models trained only on males performed poorly on females for several subtypes.
- Age Bias: The model's ability to detect the RUNX1::RUNX1T1 subtype in the elderly was significantly lower compared to younger groups when that age bracket was underrepresented. Similarly, excluding young adults from training compromised the classification of NPM1 and CBFB::MYH11 subtypes.
Limitations and Future Work
The primary limitation is the sample size; 189 individuals is relatively small for deep learning, which may exacerbate the observed biases. The authors also noted that the control group lacked elderly individuals, preventing a full bias analysis for healthy samples.
Moving forward, the researchers call for:
- Multicentric Datasets: Collecting larger, more diverse datasets from multiple institutions to improve generalizability.
- Fairness-Aware Learning: Integrating algorithmic constraints during training that explicitly penalize biased outcomes.
- Methodological Oaths: Suggesting that machine learning scientists should adopt a version of the physician's oath to prioritize fair and just medical care in their algorithmic designs.
Summary updated