FAMEWS: a Fairness Auditing tool for Medical Early-Warning Systems

Marine Hoche, Olga Mineeva, Manuel Burger, Alessandro Blasimme, Gunnar Rätsch

TL;DR

This paper introduces FAMEWS, an open-source tool designed to audit the fairness of medical early-warning systems (EWS). FAMEWS goes beyond standard fairness metrics by incorporating clinical outcome analysis and bias source screening, crucial for ensuring ethical AI in healthcare.

What Problem Does This Paper Address?

Machine learning (ML) models are increasingly used in healthcare for tasks like early detection of critical conditions. However, these models can exhibit biases, leading to unfair health outcomes for certain patient groups. While regulations are emerging to ensure ethical AI, translating these principles into practical, statistically sound fairness audits for complex medical systems like EWS remains a significant challenge. Existing fairness tools often lack the specific considerations needed for time-series data and clinical contexts, such as event-based metrics and analysis of real-world consequences.

Methods

FAMEWS provides a flexible, multi-stage fairness auditing framework tailored for medical EWS. Its key components include:

  • Flexible Patient Grouping: Allows users to define cohorts based on demographics (sex, age) or clinical attributes (admission type, comorbidities), enabling the identification of biases in subgroups beyond standard protected attributes.
  • Comprehensive Analysis Stages:
    • Model Performance Comparison: Evaluates classical fairness metrics (recall, precision, etc.) and specialized event-based and prevalence-corrected metrics across cohorts.
  • Model Outcome Comparison: Analyzes the time gap between the first correct alarm and the actual event, crucial for EWS where timing is critical.
  • Bias Source Screening: Investigates potential sources of bias by examining:
  • Label Bias: Compares medical variables used to define proxy labels across cohorts.
  • Feature Importance Bias: Assesses if feature importance rankings differ significantly across cohorts, which could lead to models that perform poorly for certain groups.
  • Missing Data Bias: Analyzes how the intensity of medical variable measurements varies across cohorts and its impact on model performance.
  • Automated PDF Reporting: Generates comprehensive, stakeholder-friendly reports that summarize audit findings, facilitating communication between developers, clinicians, and regulators.
  • Statistical Rigor: Employs bootstrapping and Mann-Whitney U tests with Bonferroni correction to ensure the statistical robustness of the findings.

Datasets

The primary dataset used for experiments and demonstrating FAMEWS is the HiRID dataset (Faltys et al., 2021), a high time-resolution ICU dataset. The authors also mention testing the framework on other alarm systems (e.g., respiratory failure detection) and datasets like MIMIC-III. The benchmark models for early-detection of organ failure used in the study were developed by Yèche et al. (2021).

Key Insights and Ideas

  • Beyond Standard Metrics: Fairness in medical EWS requires more than just comparing standard classification metrics. Analyzing clinical outcomes (like the time gap between alarm and event) and potential sources of bias (like missing data patterns) is essential.
  • Context Matters: The definition of fairness and the relevant metrics are highly context-dependent, especially in time-series medical data where events have a temporal dimension.
  • Subgroups Beyond Demographics: Identifying biases requires looking at clinically relevant subgroups (e.g., by admission type) in addition to standard demographic groups.
  • Transparency for Stakeholders: A well-structured, comprehensive report is crucial for bridging the gap between technical fairness audits and practical decision-making by clinicians, developers, and regulators.
  • Missing Data is a Major Bias Source: The intensity and patterns of missing medical data can significantly impact model performance and introduce biases across different patient groups.

Key Results

The paper presents detailed fairness audits of ML models on the HiRID dataset, showcasing FAMEWS's capabilities. While specific quantitative results are presented in the extensive tables and figures within the report appendix, the key qualitative findings include:

  • Performance Discrepancies: Significant performance differences were observed across various patient groupings (sex, age, APACHE group, surgical status) for different metrics. For instance, the study highlighted performance discrepancies for male patients and neurological patients.
  • Time Gap Disparities: The time gap between alarms and events varied significantly across cohorts, indicating potential fairness issues in how quickly different patient groups receive attention.
  • Medical Variable Distribution Differences: Distributions of key medical variables (like a_Lac and ABPm) differed notably across patient groups, suggesting potential label bias or differential measurement intensity.
  • Feature Importance Shifts: Feature rankings showed variations across cohorts, indicating that features important for the general population might not be as important for specific subgroups, potentially leading to reduced predictive power for those groups.
  • Impact of Missingness: Missingness in medical data was shown to have a significant impact on model performance, with certain metrics being substantially worse for patients with missing data.

Limitations and Future Work

  • Report Cumbersomeness: The audit report, while comprehensive, might appear cumbersome. The authors believe this is necessary for thoroughness, but acknowledge that selecting relevant results can be subjective.
  • Limited Scope of Bias Sources: The tool focuses on crucial bias sources but acknowledges that other factors might be relevant depending on the specific system.
  • Outcome Bias Measurement: Measuring outcome bias is application-dependent and might require post-deployment data.
  • Metric Selection: FAMEWS offers a curated set of metrics, including event-based and prevalence-corrected ones, but doesn't cover all possible fairness metrics.
  • Future Work: The authors encourage extending the fairness audit to post-deployment biases and exploring additional fairness metrics. They also suggest that users can adapt FAMEWS for other binary classifiers and extend its functionalities due to its open-source nature.

Summary updated

BibTeX
@inproceedings{hoche2024famews,
  author = {Marine Hoche and Olga Mineeva and Manuel Burger and Alessandro Blasimme and Gunnar Rätsch},
  title = {{FAMEWS: a Fairness Auditing tool for Medical Early-Warning Systems}},
  booktitle = {Chil},
  year = {2024},
}

Loading related content...

Figure