Prediction of carbapenem-resistant gram-negative bacterial bloodstream infection in intensive care unit based on machine learning

Qiqiang Liang, Shuo Ding, Juan Chen, Xinyi Chen, Yongshan Xu, Zhijiang Xu, Man HuangBMC Medical Informatics Decis. Mak 2024 · DOI

TL;DR

This paper introduces a machine learning approach to predict carbapenem-resistant gram-negative bacterial (CRGNB) bloodstream infections (BSIs) in intensive care units (ICUs). By analyzing patient data at the time of blood culture order, the model aims to provide early risk stratification, enabling more timely and targeted antibiotic treatment, which is crucial given the diagnostic delay of traditional blood cultures.

What Problem Does This Paper Address?

Bloodstream infections (BSIs) are a major cause of mortality in critically ill patients. A significant challenge in managing these infections, particularly in China, is the rise of multidrug-resistant bacteria (MDR), specifically carbapenem-resistant gram-negative bacteria (CRGNB). Current diagnostic methods, like blood cultures, take 2-5 days to yield results, delaying the initiation of appropriate antibiotic therapy. This delay can lead to the progression of sepsis to septic shock and death. Clinicians often resort to broad-spectrum antibiotics empirically, which can contribute to further resistance. The paper addresses the need for a predictive tool that can identify patients at high risk for CRGNB BSIs before definitive culture results are available, allowing for earlier, more informed antibiotic decisions.

Methods

The study employed a retrospective, multi-center design using data from a regional ICU database. They utilized several machine learning algorithms for prediction, including:

  • Multivariable Logistic Regression: A traditional statistical method for binary classification.
  • Decision Tree: A tree-like model where internal nodes represent features, branches represent decision rules, and leaf nodes represent outcomes.
  • Random Forest: An ensemble method that constructs multiple decision trees during training and outputs the mode of the classes (classification) or mean prediction (regression) of the individual trees.
  • Support Vector Machine (SVM): A supervised learning model that finds an optimal hyperplane to separate data points into different classes.
  • XGBoost: An optimized gradient boosting algorithm known for its speed and performance.

The dataset was split into training (70%), validation (15%), and testing (15%) sets. Model performance was evaluated using metrics such as Area Under the Receiver Operating Characteristic (AUROC) curves, accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).

Datasets

The study utilized data from the general intensive care unit (ICU) database of the Second Affiliated Hospital of Zhejiang University School of Medicine. This retrospective database contained information on approximately 18,000 critically ill patients from 2015 to 2021, including demographics, vital signs, clinical examination, medication orders, clinical diagnoses, and medical documents. For this study, 8652 effective blood cultures were analyzed, with 2375 patients identified as having BSIs. After applying exclusion criteria, 952 positive blood cultures and 1422 negative blood cultures were included for analysis.

Key Insights and Ideas

  • Early Prediction is Feasible and Crucial: The study demonstrates that machine learning can effectively predict the likelihood of a positive blood culture and, more importantly, the probability of it being caused by CRGNB, using data available at the time of the blood culture order. This is a significant step towards bridging the diagnostic gap.
  • Identification of High-Risk Factors: The research identified several key risk factors for CRGNB BSIs, including mechanical ventilation, invasive catheterization, and a history of carbapenem use. These factors are critical for risk stratification and targeted interventions.
  • Machine Learning Outperforms Traditional Methods: While logistic regression was used as a baseline, the machine learning models, particularly Random Forest and SVM, showed superior performance in predicting CRGNB BSIs, indicating their potential for more accurate and nuanced predictions.
  • Clinical Utility for Antibiotic Stewardship: The ability to predict CRGNB BSIs early can guide clinicians in selecting appropriate empirical antibiotic regimens, potentially reducing the reliance on broad-spectrum antibiotics and improving patient outcomes.

Key Results

  • Bloodstream Infection Prediction: The Random Forest model achieved the best prediction ability for BSIs with an AUROC of 0.86.
  • CRGNB Prediction: For predicting CRGNB BSIs, the SVM model achieved the highest AUROC of 0.88, closely followed by the Random Forest model with an AUROC of 0.87.
  • High-Risk Factors Identified: Multivariate analysis confirmed mechanical ventilation, invasive catheterization, and carbapenem use history as significant high-risk factors for CRGNB BSIs.
  • Patient Cohort: The study included 952 patients with positive blood cultures, of which 418 were in the CRGNB group and 534 in the non-CRGNB group.

Limitations and Future Work

The authors acknowledge several limitations:

  • Retrospective Design: The study's retrospective nature means it is subject to the inherent biases and limitations of the data collected.
  • External Adaptability: The model was developed using data from a single regional database, raising concerns about its external adaptability to different hospital settings or MDR epidemic areas. Further validation in diverse populations is needed.
  • Blood Culture Limitations: The study acknowledges the inherent limitations of blood cultures, including false positives and negatives due to contamination, varying sampling times, and early antibiotic exposure.
  • Data Completeness: While efforts were made to handle missing data, the interpretation of continuous variables might still be affected by analytical errors.

Future work suggested includes:

  • Prospective and Randomized Controlled Trials: To confirm the model's efficacy and guide clinical practice, prospective studies and randomized controlled trials are necessary.
  • Integration of New Technologies: Exploring the inclusion of data from newer diagnostic technologies like Next-Generation Sequencing (NGS) and Polymerase Chain Reaction (PCR) could potentially enhance diagnostic accuracy.
  • Model Optimization and Scenario Application: Continuous improvement of the model through increased data volume and optimization, along with testing in real-world clinical scenarios, is crucial for its practical application.
  • External Validation and Model Adaptation: Developing strategies for external validation and adapting the model framework for different epidemic areas will be essential for broader utility.

Summary updated

BibTeX
@article{liang2024prediction,
  author = {Qiqiang Liang and Shuo Ding and Juan Chen and Xinyi Chen and Yongshan Xu and Zhijiang Xu and Man Huang},
  title = {{Prediction of carbapenem-resistant gram-negative bacterial bloodstream infection in intensive care unit based on machine learning}},
  journal = {BMC Medical Informatics Decis. Mak.},
  year = {2024},
  doi = {10.1186/s12911-024-02504-4},
}
Figure