FairEHR-CLP: Towards Fairness-Aware Clinical Predictions with Contrastive Learning in Multimodal Electronic Health Records

Yuqing Wang, Yuqing Wang 0004, Malvika Pillai, Yun Zhao, Catherine M. Curtin, Tina Hernandez-Boussard

TL;DR

This paper introduces FairEHR-CLP, a novel framework that uses contrastive learning on multimodal Electronic Health Records (EHRs) to make clinical predictions fairer across demographic groups. By generating synthetic patient counterparts and aligning representations, it significantly reduces bias without sacrificing predictive accuracy, addressing a critical need for equitable healthcare AI.

What Problem Does This Paper Address?

Electronic Health Records (EHRs) hold immense potential for improving healthcare through predictive modeling. However, EHR data often reflects and perpetuates societal biases related to race, gender, age, and socioeconomic status. If not addressed, machine learning models trained on this data can lead to unfair outcomes, exacerbating existing health disparities. Existing fairness-enhancing methods often struggle with the multimodal nature of EHRs (combining clinical notes, lab results, etc.) and the complex interplay of various demographic factors. This paper tackles the challenge of developing a fair prediction model that leverages the richness of multimodal EHR data while actively mitigating these intertwined social biases.

Methods

The core of FairEHR-CLP is a two-stage approach:

  1. Synthetic Counterpart Generation: For each real patient, synthetic counterparts are created. These synthetic patients share similar health conditions but have different demographic attributes (gender, race, ethnicity, age, insurance type). This is achieved using EHR-based Generative Adversarial Networks (GANs) for longitudinal data and a large language model (Llama2-70b) for clinical notes. This stage aims to create a more diverse representation of patient demographics while preserving health information.
  2. Fairness-Aware Predictions with Contrastive Learning (CL): In this stage, the model learns to align the representations of real patients with their synthetic counterparts. Contrastive learning is used to minimize the distance between representations of similar health conditions across different demographics. This is combined with a multi-layer perceptron (MLP) classifier for the actual prediction task. A novel fairness metric, the Error Distribution Disparity Index (EDDI), is also introduced to better measure fairness in settings with imbalanced data and varying group sizes.

Datasets

The authors evaluated their framework on three large-scale, publicly available EHR datasets:

  • STARR: From Stanford Medicine.
  • MIMIC-III: A widely used critical care database.
  • MIMIC-IV: The successor to MIMIC-III.

These datasets were used for three binary classification tasks: predicting delirium, opioid use disorder (OUD), and 30-day readmission, all of which have significant implications for patient care.

Key Insights and Ideas

  • Synthetic Data for Fairness: Generating synthetic patient data with varied demographics but similar health profiles is a powerful way to train models that are less sensitive to protected attributes. It allows the model to learn underlying health patterns independent of demographic correlations.
  • Contrastive Learning for Bias Mitigation: CL is not just for representation learning; it can be effectively repurposed to align representations across sensitive attributes, directly addressing bias in a multimodal setting.
  • Multimodality is Key: Combining demographics, longitudinal data, and clinical notes provides a more comprehensive patient picture, leading to both better predictive performance and improved fairness compared to using subsets of data.
  • New Fairness Metric Matters: The proposed EDDI metric is crucial for evaluating fairness in real-world clinical scenarios, which often involve imbalanced classes and varying subgroup sizes, where traditional metrics might fall short.

Key Results

  • FairEHR-CLP consistently outperformed baseline methods in both predictive accuracy (F1 and AUROC) and fairness metrics (EO and EDDI) across multiple datasets and tasks.
  • Compared to a demographic-free baseline (DfC), FairEHR-CLP achieved comparable fairness while significantly improving predictive accuracy by an average of 4.8% in F1 and 5.8% in AUROC.
  • When compared to specialized debiasing methods like Adversarial Debiasing and Fair Patient Model, FairEHR-CLP showed superior performance in both accuracy and fairness.
  • Ablation studies confirmed that using all data modalities (demographics, longitudinal, and notes) yielded the best results for both performance and fairness. Removing contrastive learning or the dynamic relevance layer led to a significant drop in performance and increased bias.
  • Visualizations (t-SNE) showed that the full FairEHR-CLP model produced more homogeneous representations across racial groups, indicating a reduced reliance on race for predictions.

Limitations and Future Work

The authors acknowledge that the quality of synthetic data is a potential limitation. Inaccuracies in capturing the full complexity of real patient data could affect the model's bias mitigation effectiveness. Future work could explore more advanced synthetic data generation techniques, especially for longitudinal data and clinical notes. Additionally, handling ambiguous categories in sensitive attributes (e.g., "Unknown") needs further refinement. The authors also plan to extend their experiments to various clinical contexts to enhance the framework's robustness and adaptability.

Summary updated

Abstract

In the high-stakes realm of healthcare, ensuring fairness in predictive models is crucial. Electronic Health Records (EHRs) have become integral to medical decision-making, yet existing methods for enhancing model fairness restrict themselves to unimodal data and fail to address the multifaceted social biases intertwined with demographic factors in EHRs. To mitigate these biases, we present FairEHR-CLP: a general framework for Fairness-aware Clinical Predictions with Contrastive Learning in EHRs. FairEHR-CLP operates through a two-stage process, utilizing patient demographics, longitudinal data, and clinical notes. First, synthetic counterparts are generated for each patient, allowing for diverse demographic identities while preserving essential health information. Second, fairness-aware predictions employ contrastive learning to align patient representations across sensitive attributes, jointly optimized with an MLP classifier with a softmax layer for clinical classification tasks. Acknowledging the unique challenges in EHRs, such as varying group sizes and class imbalance, we introduce a novel fairness metric to effectively measure error rate disparities across subgroups. Extensive experiments on three diverse EHR datasets on three tasks demonstrate the effectiveness of FairEHR-CLP in terms of fairness and utility compared with competitive baselines. FairEHR-CLP represents an advancement towards ensuring both accuracy and equity in predictive healthcare models.
BibTeX
@inproceedings{wang2024fairehr,
  author = {Yuqing Wang and Yuqing Wang and Malvika Pillai and Yun Zhao and Catherine M. Curtin and Tina Hernandez-Boussard},
  title = {{FairEHR-CLP: Towards Fairness-Aware Clinical Predictions with Contrastive Learning in Multimodal Electronic Health Records}},
  booktitle = {Mlhc},
  year = {2024},
}
Figure