Personalising intravenous to oral antibiotic switch decision making through fair interpretable machine learning

W. Bolton, Richard Wilson, Mark Gilchrist, Pantelis Georgiou, Alison Holmes, Timothy M. RawsonNature Communications 2024 · DOI

TL;DR

This paper introduces a machine learning model that predicts when critically ill patients can safely switch from intravenous (IV) to oral antibiotics. The model is designed to be fair, interpretable, and personalized, offering clinicians a decision support tool to optimize antibiotic use, combat antimicrobial resistance, and improve patient safety.

What Problem Does This Paper Address?

Antimicrobial stewardship is crucial for combating resistance and healthcare-associated infections. A key aspect is optimizing antibiotic therapy by switching from intravenous (IV) to oral administration when appropriate. However, this decision is often complex and not standardized, leading to delayed switches or unnecessary IV use. This can increase the risk of IV-related complications, patient discomfort, and healthcare costs. While guidelines exist, a lack of robust decision support systems hinders widespread adoption of early oral switching. This paper aims to bridge this gap by developing a machine learning-based clinical decision support system (CDSS) to personalize IV-to-oral switch decisions using routinely collected clinical data.

Methods

The researchers developed a machine learning model using neural networks. They employed a two-stage feature selection process: first, using SHAP values to identify important features from an initial large set, and then using a genetic algorithm to further refine these into two sets: a "short" set of 5 features and a "long" set of 37 features. The Canonical Time-series Characteristics (Catch22) methodology was used to transform temporal clinical data into daily tabular values. Model performance was evaluated using metrics like AUROC, accuracy, precision, TPR, and FPR. To ensure fairness, they assessed model performance across various sensitive attribute groups and employed threshold optimization techniques to achieve "equalized odds." Interpretability was achieved using the SimplEx methodology, generating visual explanations to help clinicians understand the model's predictions.

Datasets

The study utilized two large, publicly available, de-identified real-world clinical datasets:

  • MIMIC-IV: This dataset contains over 40,000 patient admissions to the Beth Israel Deaconess Medical Center between 2008 and 2019. It was used for feature selection, model optimization, and hold-out testing.
  • eICU Collaborative Research Database: This dataset includes data from over 200,000 ICU admissions across the United States from 2014 to 2015. It was used for transfer learning to assess the generalizability of the developed models.

Key Insights and Ideas

  • Clinical Data is Sufficient: Routinely collected clinical parameters, even without specialized lab tests, can be powerful predictors for IV-to-oral switch decisions. This highlights the potential for leveraging existing EHR data.
  • Simplicity vs. Performance Trade-off: A "short" feature set (5 features) achieved comparable performance to a "long" feature set (37 features) in many scenarios, suggesting that simpler models can be highly effective and potentially easier to implement and understand.
  • Interpretability is Key for Adoption: The focus on creating clinically useful visual explanations (e.g., traffic light systems, feature importance) is crucial for building trust and facilitating the adoption of AI in clinical practice.
  • Fairness is Achievable, but Requires Effort: While the models generally performed fairly across sensitive attribute groups, achieving "equalized odds" sometimes required specific threshold optimization, demonstrating that fairness is an active design consideration, not an automatic outcome.
  • Temporal Dynamics Matter: The use of time-series features (like those derived from O2 saturation and blood pressure over time) proved critical, reflecting the dynamic nature of patient recovery and the importance of observing trends rather than single data points.

Key Results

  • The best performing model achieved a mean AUROC of 0.80 (SD 0.01) on the hold-out set.
  • The models demonstrated good performance across various patient subgroups, including those with specific infections like sepsis and pneumonia.
  • The developed CDSS was shown to be fair, with performance not significantly biased against protected characteristics, although some minor discrepancies required further optimization.
  • The interpretability methods provided clinically relevant visual explanations, aiding in understanding the model's decision-making process.
  • Transfer learning to the eICU dataset showed promising generalizability, with AUROC scores around 0.72 for the short model.

Limitations and Future Work

  • Historical Data Bias: The models are trained on historical data, which may reflect past prescribing practices that are now considered suboptimal due to evolving AMR concerns.
  • Snapshot vs. Holistic View: The model analyzes daily snapshots and may not capture all nuanced clinical factors considered by physicians.
  • US-Centric Data: The current models are trained on US ICU data, and their performance in other healthcare settings, countries, or non-ICU environments needs further validation.
  • Prospective Evaluation Needed: The authors emphasize that prospective evaluation of safety and efficacy is essential before clinical deployment.
  • Integration of Logic Rules: Future work could involve integrating logic-based rules (e.g., for specific infection types, malabsorption, or patient compliance issues) to enhance the CDSS.
  • Real-time Implementation: Developing real-time implementation of such CDSSs for continuous decision support is a key future direction.

Summary updated

Abstract

Antimicrobial resistance (AMR) and healthcare associated infections pose a significant threat globally. One key prevention strategy is to follow antimicrobial stewardship practices, in particular, to maximise targeted oral therapy and reduce the use of indwelling vascular devices for intravenous (IV) administration. Appreciating when an individual patient can switch from IV to oral antibiotic treatment is often non-trivial and not standardised. To tackle this problem we created a machine learning model to predict when a patient could switch based on routinely collected clinical parameters. 10,362 unique intensive care unit stays were extracted and two informative feature sets identified. Our best model achieved a mean AUROC of 0.80 (SD 0.01) on the hold-out set while not being biased to individuals protected characteristics. Interpretability methodologies were employed to create clinically useful visual explanations. In summary, our model provides individualised, fair, and interpretable predictions for when a patient could switch from IV-to-oral antibiotic treatment. Prospectively evaluation of safety and efficacy is needed before such technology can be applied clinically.
BibTeX
@inproceedings{bolton2024personalising,
  author = {W. Bolton and Richard Wilson and Mark Gilchrist and Pantelis Georgiou and Alison Holmes and Timothy M. Rawson},
  title = {{Personalising intravenous to oral antibiotic switch decision making through fair interpretable machine learning}},
  booktitle = {Nature Communications},
  year = {2024},
  doi = {10.1038/s41467-024-44740-2},
}
Figure