Algorithmic Fairness in Chest X-ray Diagnosis: A Case Study

Haoran Zhang, Thomas Hartvigsen, Marzyeh Ghassemi2023 · DOI

Abstract

Machine learning models are being increasingly deployed in real-world clinical environments. However, these models often exhibit disparate performance between population groups, potentially leading to inequitable and discriminatory predictions. In this case study, we use several distinct concepts of algorithmic fairness to analyze a deep learning model that predicts from their chest X-ray whether someone has a disease. After observing disparities in the false positive rate and false negative rate between groups from several protected classes, we apply algorithmic fairness methods to remove such disparities. However, we find that such algorithmic interventions can have serious unintended consequences. Finally, we question what the appropriate definition of fairness is in the clinical context, and advocate for an investigation of bias in the data whenever possible, as opposed to blindly applying algorithmic interventions.

Introduction: The Problem of Underdiagnosis Bias

As deep learning models for medical imaging reach specialist-level performance, they are increasingly deployed in clinical settings. However, these models often exhibit "underdiagnosis bias"—a higher rate of false positives for the "No Finding" label. This means the AI incorrectly predicts a patient is healthy when they actually have a pathology, potentially delaying critical care.

This study investigates why these disparities exist across protected classes—including race, sex, age, and insurance type—and evaluates whether common algorithmic fairness techniques effectively mitigate these issues. A central tension explored is the conflict between Group Fairness (equalizing performance across groups) and Clinical Utility (ensuring the best possible care for every individual).

Methodology and Data

The researchers trained a deep convolutional neural network (CNN) to predict the absence of disease ("No Finding") from chest X-rays using the MIMIC-CXR dataset. This public resource contains over 300,000 images and associated radiology reports. To extract labels from free-text reports, the study utilized the CheXpert labeler, an automated NLP tool.

To address observed unfairness, three main approaches were compared:

  1. Post-hoc Thresholding: Adjusting decision thresholds for different demographic groups to equalize False Positive Rates (FPR) and False Negative Rates (FNR).
  2. In-training Debiasing: Incorporating fairness terms (e.g., MMDMatch, Adversarial training) into the loss function to force the model to ignore demographic features.
  3. Label Audit: Comparing automated CheXpert labels against a "gold standard" of 1,200 manually verified reports by a board-certified radiologist to identify underlying "label bias."

Performance Disparities and the Cost of Fairness

The baseline model revealed significant disparities. Female, younger, Black, and Medicaid-insured patients all showed significantly higher rates of underdiagnosis (FPR for "No Finding"). Furthermore, False Negative Rates (FNR) varied widely across age and racial subgroups.

Subgroup and intersectional False Negative Rates (FNR) demonstrating performance disparities across age, sex, race, and insurance type.

When applying algorithmic interventions to achieve "Equal Odds," the researchers observed a "leveling down" effect: enforcing group fairness constraints often resulted in a model that performed worse for all groups compared to the original unconstrained model. In a clinical setting, a definition of fairness that harms everyone is ethically problematic.

Intersectional False Positive Rates (FPR) for female patients, illustrating the 'underdiagnosis bias' across demographic intersections.

Root Causes: Label Bias and Calibration

The study suggests that disparities are frequently rooted in the training data itself. The CheXpert labeler was found to be significantly less accurate for patients aged 80 and older (only 64.1% accurate for "No Finding"). Because the "ground truth" the model learns from is noisier for the elderly, the model naturally performs worse on that demographic.

Additionally, the authors argue that differences in calibration—whether a predicted risk corresponds to real-world probability—are a more significant source of disparity than simple error rate gaps. They advocate for Minimax Pareto Fairness, which focuses on maximizing the performance of the worst-case group rather than forcing mathematical parity at the expense of overall accuracy.

Conclusion

Algorithmic fairness is only one component of equitable healthcare; bias can also be introduced downstream by how clinicians use AI recommendations. This study concludes that we cannot simply "algorithm" our way out of biased data. Instead, we must prioritize auditing data quality, fixing label bias, and choosing fairness definitions that prioritize the best possible outcomes for the most vulnerable patients.

Summary updated

Abstract

Machine learning models are being increasingly deployed in real-world clinical environments. However, these models often exhibit disparate performance between population groups, potentially leading to inequitable and discriminatory predictions. In this case study, we use several distinct concepts of algorithmic fairness to analyze a deep learning model that predicts from their chest X-ray whether someone has a disease. After observing disparities in the false positive rate and false negative rate between groups from several protected classes, we apply algorithmic fairness methods to remove such disparities. However, we find that such algorithmic interventions can have serious unintended consequences. Finally, we question what the appropriate definition of fairness is in the clinical context, and advocate for an investigation of bias in the data whenever possible, as opposed to blindly applying algorithmic interventions.
BibTeX
@misc{zhang2023algorithmic,
  author = {Haoran Zhang and Thomas Hartvigsen and Marzyeh Ghassemi},
  title = {{Algorithmic Fairness in Chest X-ray Diagnosis: A Case Study}},
  year = {2023},
  doi = {10.21428/2c646de5.816cae0e},
}
Figure