An adversarial training framework for mitigating algorithmic biases in clinical machine learning

Jenny Yang, Andrew A. S. Soltan, David W. Eyre, David A. CliftonNpj Digit. Medicine 2023 · OpenReview

TL;DR

This paper introduces an adversarial training framework to mitigate biases in clinical machine learning models, specifically for COVID-19 prediction. By training a predictor network against an adversary network, they successfully reduced biases related to patient ethnicity and hospital location while maintaining high predictive performance, demonstrating a practical approach to fairer AI in healthcare.

What Problem Does This Paper Address?

Machine learning models, while powerful, can inherit and even amplify existing biases present in healthcare data. This can lead to unfair outcomes, where certain patient groups receive poorer care or less accurate diagnoses. These biases can stem from various sources, including how data is collected, processed, and represented across different hospitals or demographic groups. For instance, historical biases in medical practice have shown disparities in how symptoms are treated based on gender or ethnicity. Similarly, variations in healthcare practices and patient populations across different hospitals can introduce "site-specific" biases. The challenge is to develop models that are not only accurate but also fair, ensuring equitable performance across all patient populations and healthcare settings.

Methods

The core of this paper is an adversarial debiasing framework. This approach involves training two neural networks simultaneously:

  1. Predictor Network (P): This network is trained to perform the primary task, in this case, predicting COVID-19 status from clinical features.
  2. Adversary Network (A): This network is trained to detect and exploit biases in the predictor network's outputs. It tries to predict the sensitive attribute (e.g., ethnicity, hospital location) based on the predictor's predictions.

The two networks are trained in opposition. The predictor network aims to minimize its prediction error (for COVID-19 status) while simultaneously maximizing the adversary's error (i.e., making it difficult for the adversary to predict the sensitive attribute). This forces the predictor to learn representations that are independent of the sensitive attributes.

A modified loss function is used to ensure stable training, especially at the beginning. This function balances the predictor's task loss with the adversary's loss, with an adjustable hyperparameter controlling the strength of the debiasing effort. The authors also detail their approach to hyperparameter optimization using cross-validation and threshold optimization to achieve desired sensitivity levels for clinical utility.

Datasets

The study utilized clinical data from four UK National Health Service (NHS) Trusts:

  • Oxford University Hospitals NHS Foundation Trust (OUH)
  • University Hospitals Birmingham NHS Trust (UHB)
  • Bedfordshire Hospitals NHS Foundations Trust (BH)
  • Portsmouth Hospitals University NHS Trust (PUH)

Data included de-identified demographic information and routinely collected clinical features (vital signs, blood tests, liver function tests, urea & electrolytes). The data was split into training, validation, and held-out test sets. For ethnicity debiasing, data was primarily from OUH. For hospital debiasing, data from all four trusts was combined. This multi-site dataset is a key aspect, allowing for both prospective and external validation.

Key Insights and Ideas

  • Adversarial Debiasing is Effective in Clinical Settings: The study demonstrates that adversarial debiasing can successfully mitigate biases related to sensitive attributes like ethnicity and hospital location in a real-world clinical prediction task. This moves beyond theoretical applications to practical healthcare scenarios.
  • Fairness Doesn't Necessarily Mean Performance Loss: A crucial finding is that improving fairness (measured by equalized odds) did not significantly compromise the model's predictive performance (AUROC, sensitivity, specificity, NPV). This addresses a common concern that fairness interventions might degrade accuracy.
  • Site-Specific Biases are Quantifiable and Mitigatable: The t-SNE visualization clearly showed distinct clusters for different hospitals, highlighting how site-specific features can embed biases. The adversarial approach effectively addressed these, showing that hospital location is a significant source of bias that can be managed.
  • Multiclass Fairness is Achievable: Unlike much prior work focusing on binary sensitive attributes, this study successfully applied adversarial debiasing to multiclass attributes (ethnicity categories and hospital locations), demonstrating broader applicability.
  • Data Imbalance is a Key Challenge: The authors acknowledge that imbalanced datasets (e.g., skewed ethnicity representation) can impact the effectiveness of debiasing, suggesting that balanced data would further improve results.

Key Results

  • Ethnicity Debiasing: Adversarial models achieved improved equalized odds across external test cohorts, with minimal trade-offs in AUROC. For example, in the BH cohort, AUROC was 0.894 for both basic and adversarial models. Equalized odds metrics (TP SD and FP SD) were demonstrably improved by the adversarial approach.
  • Hospital Debiasing: For the combined sites dataset, adversarial models also showed improved equalized odds performance. AUROC scores remained high and comparable to basic models (0.905 for basic vs. 0.902 for adversarial). Negative Predictive Values (NPVs) remained very high (>0.99) for both, indicating strong ability to rule out COVID-19.
  • Clinical Utility: All models achieved clinically-effective screening performance, with Negative Predictive Values (NPVs) consistently above 0.98, meaning they could reliably identify patients unlikely to have COVID-19.

Limitations and Future Work

  • Data Imbalance: The study acknowledges that the training datasets were imbalanced with respect to sensitive features. They suggest that balanced data would likely lead to stronger debiasing effects.
  • Ambiguous Ethnicity Categories: The use of broad categories like "Unknown," "Other," and "Mixed" for ethnicity might have hindered the adversary's ability to learn distinct biases, suggesting that more specific labels would be beneficial.
  • Generalizability: While validated across multiple NHS trusts, the datasets were exclusively from the UK. Future work could explore domain adaptation and transfer learning to improve generalizability to diverse global populations.
  • Threshold Optimization: The authors note that while they optimized thresholds for sensitivity, site-specific thresholds might be necessary for optimal performance and fairness across different hospitals due to varying data distributions.
  • Missing Data: The imputation of missing values might have masked important information or biases. Future research could explore alternative methods for handling missing data.
  • Definition of Fairness: The study focused on "equalized odds." Future work could explore other fairness metrics and their trade-offs.

Summary updated

Abstract

Machine learning is becoming increasingly prominent in healthcare. Although its benefits are clear, growing attention is being given to how these tools may exacerbate existing biases and disparities. In this study, we introduce an adversarial training framework that is capable of mitigating biases that may have been acquired through data collection. We demonstrate this proposed framework on the real-world task of rapidly predicting COVID-19, and focus on mitigating site-specific (hospital) and demographic (ethnicity) biases. Using the statistical definition of equalized odds, we show that adversarial training improves outcome fairness, while still achieving clinically-effective screening performances (negative predictive values >0.98). We compare our method to previous benchmarks, and perform prospective and external validation across four independent hospital cohorts. Our method can be generalized to any outcomes, models, and definitions of fairness.
BibTeX
@inproceedings{yang2023adversarial,
  author = {Jenny Yang and Andrew A. S. Soltan and David W. Eyre and Yang Yang and David A. Clifton},
  title = {{An adversarial training framework for mitigating algorithmic biases in clinical machine learning}},
  booktitle = {Npj Digit. Medicine},
  year = {2023},
  url = {https://openreview.net/forum?id=8cMHicKKXw},
}
Figure