TL;DR
Researchers developed an automated framework to audit racial bias in machine learning models used to identify patients with Alzheimer’s and related dementias. They found that while some disease models are stable across groups, others show performance swings as high as 30%, proving that "race-blind" models can still harbor significant subgroup disparities.
What Problem Does This Paper Address?
In modern medicine, we often use "phenotyping algorithms" to scan Electronic Health Records (EHRs) and identify cohorts of patients with specific conditions. Traditionally, these were rigid, rule-based systems (e.g., "If patient has Code X and Medication Y, they have Disease Z"). Recently, the field has shifted toward probabilistic phenotyping—using machine learning to assign a probability score to patients.
While ML models are more flexible and can handle missing data better than rules, they are "black boxes" prone to inheriting biases from the underlying data. This paper addresses a critical gap: we don't fully understand how these probabilistic models perform across different racial subgroups, especially for high-stakes conditions like Alzheimer’s Disease and Related Dementias (ADRD) in older adults. If an algorithm is 90% accurate for one group but only 60% for another, it could exacerbate existing health disparities.
Methods
The authors leveraged the APHRODITE framework, which uses "weak supervision" to build phenotype models. Instead of needing thousands of manually labeled charts, APHRODITE uses high-precision keywords to find a "seed" group and then trains a classifier to find similar patients.
The researchers implemented a multi-pronged experimental framework to hunt for bias:
- Model Selection: They tested three common algorithms: LASSO (logistic regression with regularization), Random Forests, and Support Vector Machines (SVM).
- Stratified Evaluation: They built models using different data compositions:
- Traditional: All races merged.
- Balanced: Equal distribution of patients from each race.
- Single-race only: Training on one race and testing on all others.
- Leave-one-out: Systematically removing one race from the training set.
- Metrics: Beyond standard accuracy, they focused on variation—measuring the delta in performance when the model encounters a subgroup it wasn't primarily trained on.
Datasets
The study utilized the Stanford Medicine Research data Repository (STARR), a massive de-identified dataset containing clinical data for over 3.1 million patients from 2008 to 2018. The data was standardized using the OMOP Common Data Model (CDM), which allows the same code to run across different hospital systems.
The analysis focused on four specific phenotypes:
- Dementia
- Mild Cognitive Impairment (MCI)
- Alzheimer’s disease
- Parkinson’s disease
Key Insights and Ideas
- "Race-Blind" is not "Bias-Free": Even though race was not used as an input feature for the models, significant performance gaps emerged. This suggests that other clinical features (comorbidities, frequency of visits, or specific coding patterns) act as proxies for race.
- Phenotype-Specific Volatility: Bias isn't a universal constant. Models for Dementia and Alzheimer’s were surprisingly stable, with less than 8% variance across races. However, Parkinson’s and MCI models were highly sensitive, showing performance drops of up to 30%.
- The "Data Dominance" Effect: Removing the majority group (White patients) from the training data caused the most significant performance crashes. This highlights how models "overfit" to the majority's clinical patterns, making them less generalizable to minority groups.
- Underrepresentation is a Generalization Killer: Groups with very low representation (like Native Americans in this dataset, <0.25%) resulted in models that could not generalize at all, particularly regarding sensitivity (the ability to correctly identify positive cases).
Key Results
- General Accuracy: Most "full" models (trained on all data) achieved 70% to 90% accuracy.
- The Variance Gap: For Parkinson’s disease and MCI, the variance in classification accuracy reached 27-30% in some scenarios. This effectively turns a sophisticated ML model into something barely better than a coin flip for certain subgroups.
- Sensitivity Swings: Sensitivity (recall) was often more affected than overall accuracy. In the "leave-one-out" scenarios for Parkinson’s, sensitivity variance was frequently in the double digits, meaning the model was much more likely to "miss" a diagnosis in specific racial groups.
Limitations and Future Work
The authors identify several hurdles:
- Single-Site Study: The data comes only from Stanford. Clinical coding practices vary by institution, so these specific biases might look different elsewhere.
- Noisy Labels: Race is self-reported and was missing for roughly 23% of the cohort. If the "ground truth" for race is messy, the bias measurements will be too.
- Algorithm Scope: The study only looked at three "classical" ML algorithms. Future work could explore if Deep Learning architectures mitigate or exacerbate these variances.
- The "Fix" is Non-Trivial: The authors warn that simply "fixing" a model for one subgroup can sometimes degrade performance for everyone else. They suggest exploring federated learning or ensemble models as potential solutions.
Summary updated