Abstract
This study investigates the impact of demographic biases—specifically age and sex—on the performance of Multiple Instance Learning (MIL) models used for classifying Acute Myeloid Leukemia (AML) subtypes. By systematically varying the representation of different demographic groups in training data, the researchers demonstrate that underrepresentation can lead to significant diagnostic disparities. The findings highlight a "female advantage" in certain genetic subtypes and a pronounced performance gap for elderly patients, emphasizing the critical need for fairness-aware development in medical AI.
Introduction
In medical artificial intelligence, algorithmic fairness is a clinical necessity. Disparities in model performance across demographic groups can lead to inequitable patient care. Acute Myeloid Leukemia (AML) is a genetically diverse blood cancer where precise subtype identification is essential for treatment selection. Recent advancements utilize Multiple Instance Learning (MIL) to analyze thousands of single-cell images from blood smears. However, the potential for demographic bias in these models remains under-explored. This paper examines how patient age and sex in training sets influence model generalization and diagnostic accuracy.
Methodology
The researchers implemented a two-stage deep learning pipeline:
- Feature Extraction: A ResNet34 model, pre-trained on over 300,000 single-cell images, was used to convert individual cell images into high-dimensional feature vectors.
- Classification: An Attention-based MIL architecture was employed. This model processes a "bag" of cells from a single patient, using an attention mechanism to identify and weigh the cells most indicative of specific AML subtypes.
To evaluate bias, two experimental frameworks were established:
- Sex Bias Experiments: Models were trained with varying male-to-female ratios (0%, 25%, 50%, 75%, and 100% female) and evaluated on sex-specific test sets.
- Age Bias Experiments: Using a "leave-one-group-out" strategy, models were trained on four age cohorts and tested on the fifth (e.g., training on all but the "Elderly" group) to assess performance on unseen age ranges.
Dataset Characteristics
The study utilized a curated dataset of 81,214 single-cell images from 189 individuals. The cohort included four AML genetic subtypes—PML::RARA, NPM1, CBFB::MYH11, and RUNX1::RUNX1T1—alongside a healthy control group. Patients ranged from 19 to 86 years old, with a sex distribution of approximately 60% male and 40% female.
Impact of Sex Bias on Classification
The experiments revealed that sex representation in training data significantly affects classification performance, measured by the Area Under the Precision-Recall Curve (AUPRC). A notable finding was the "female advantage" for the PML::RARA subtype: increasing female representation in the training set improved accuracy for both male and female test sets. Conversely, performance on female test sets was highly sensitive to imbalance, dropping sharply when female representation was low.
Impact of Age Bias on Classification
Age-related bias was most prominent in the elderly cohort (72–86 years). When this group was excluded from training, the model struggled to generalize, particularly for the RUNX1::RUNX1T1 subtype. This performance gap may be attributed to lower sample quality in older patients and the relative rarity of certain mutations in that age group, creating a significant diagnostic challenge for AI models trained on younger populations.
Key Results and Discussion
- Subtype Sensitivity: While "Healthy Control" identification remained consistently high regardless of bias, genetic subtypes like
RUNX1::RUNX1T1required balanced training sets (50/50 sex ratio) for optimal performance. - Generalization Disparities: The study confirms that models do not automatically generalize across demographic boundaries. Features learned from one group (e.g., young males) may not fully capture the biological variance present in another (e.g., elderly females).
- Limitations: Data scarcity for specific subtype-age combinations and inherent biological factors (like cell quality) remain confounding challenges.
Conclusion
This research underscores that demographic representation is not merely a statistical detail but a fundamental component of model validity in hematopathology. Future work must prioritize the use of diverse, multicentric datasets and the integration of fairness-aware learning techniques to ensure that AI-driven diagnostics are reliable for all patient populations.
Summary updated