Machine learning for the prediction of in-hospital mortality in patients with spontaneous intracerebral hemorrhage

Baojie Mao, Rui Zhang, Yuhang Pan, Ruzi Zheng, Yanfei Shen, Wei Lü, Yuning Lu, Shanhu Xu, Jiong Wu, Ming Wang, Shu WanmedRxiv 2023 · DOI

This study presents a machine learning approach to predicting in-hospital mortality for patients suffering from spontaneous intracerebral hemorrhage (sICH). By leveraging large-scale clinical data and the XGBoost algorithm, the researchers developed a highly accurate and interpretable tool designed to help ICU physicians identify high-risk patients within the first 24 hours of admission.

TL;DR

Researchers developed an XGBoost-based model to predict in-hospital mortality for patients with spontaneous intracerebral hemorrhage (sICH) in the ICU. By identifying 14 key clinical variables, the tool achieves high predictive accuracy (AUC = 0.907) and offers an interpretable, web-based interface for real-time clinical decision support.

What Problem Does This Paper Address?

Spontaneous intracerebral hemorrhage—a sudden bleed within the brain tissue—is a critical neurological emergency with a high mortality rate, often exceeding 40%. For clinicians in the Intensive Care Unit (ICU), the first 24 hours are a "golden window" for intervention. However, predicting which patients will survive is notoriously difficult due to the complex, non-linear nature of clinical data.

While traditional scoring systems exist, they often lack the flexibility to handle the high-dimensional data found in modern electronic health records. This paper seeks to fill that gap by using machine learning (ML) to create a robust, early-warning system that generalizes across different hospital systems.

Methods

The researchers followed a rigorous pipeline to move from raw data to a deployable model:

  1. Feature Selection: They started with 46 candidate variables and used LASSO (Least Absolute Shrinkage and Selection Operator) regression to narrow them down to the 14 most impactful predictors. LASSO is a regularization technique that "shrinks" the coefficients of less important features to zero, effectively performing automated feature selection.
  2. Model Comparison: They trained and compared five distinct algorithms: Logistic Regression (LR), K-Nearest Neighbors (KNN), Adaptive Boosting (AdaBoost), Random Forest (RF), and eXtreme Gradient Boosting (XGBoost).
  3. Interpretability: To move beyond the "black box" nature of ML, they employed SHAP (SHapley Additive exPlanations). This game-theoretic approach assigns each feature an importance value for a particular prediction, allowing clinicians to see why the model flagged a patient as high-risk.
  4. Deployment: The final model was hosted as a web-based calculator to allow for real-time bedside use.

Datasets

The study utilized two distinct data sources:

  • MIMIC-IV: A massive, open-access critical care database from Beth Israel Deaconess Medical Center in Boston. This served as the primary dataset for training and internal validation ($N=1,486$).
  • Zhejiang Hospital Database: A local dataset from China used for external validation ($N=110$). Using an external cohort from a different country is a crucial step in proving the model's "real-world" utility and generalizability.

Key Insights and Ideas

  • The "Big Five" Predictors: While 14 variables were used, the SHAP analysis revealed that the Glasgow Coma Scale (GCS), SOFA score (organ failure), anticoagulant use, mannitol use, and oxygen saturation were the most influential.
  • The Mannitol Paradox: Interestingly, the model suggested that the use of mannitol (a standard treatment to reduce intracranial pressure) was associated with a poorer prognosis in this cohort. The authors suggest this might be because mannitol is often a "last-resort" treatment for the most severe cases, or that hypertonic saline might be a more effective alternative.
  • Anticoagulants and Survival: The study found that the use of anticoagulants was associated with lower mortality. This remains a controversial topic in sICH research, but the authors hypothesize it may reduce complications like thromboembolism without significantly increasing re-bleeding risk in certain ICU contexts.
  • XGBoost Supremacy: Among the five models, XGBoost consistently outperformed the others, particularly in its ability to handle the non-linear relationships between vital signs and outcomes.

Key Results

  • High Discriminative Power: In the internal validation set, the XGBoost model achieved an Area Under the Curve (AUC) of 0.907, significantly outperforming traditional logistic regression (AUC = 0.851).
  • External Robustness: Even when tested on the geographically and demographically different Zhejiang dataset, the model maintained a respectable AUC of 0.788.
  • Clinical Utility: The researchers determined an optimal prediction threshold of 29.53%. If the model predicts a mortality risk higher than this, it serves as a high-priority trigger for clinical intervention.

Limitations and Future Work

Despite the strong results, the authors acknowledge several hurdles:

  • Missing Radiologic Data: The model does not currently include hematoma volume or location—two of the most powerful predictors in neurology—because this data was not consistently structured in the administrative databases.
  • Retrospective Design: The study looks at historical data. To truly prove its worth, the model needs to be tested in a prospective clinical trial where it is used in real-time to guide treatment.
  • Sample Size: The external validation cohort ($N=110$) was relatively small compared to the training set, suggesting a need for larger multi-center studies to further refine the algorithm.

Summary updated

Abstract

Backgrounds Early and accurate identification of patients with spontaneous intracerebral hemorrhage(sICH) who are at high risk of in-hospital death can help intensive care unit (ICU) physicians make optimal clinical decisions. The aim of this study was to develop a machine learning(ML)-based tool to predict the risk of in-hospital death in patients with sICH in ICU. Methods We conducted a retrospective administrative database study using the MIMIC-IV and Zhejiang Hospital database. The outcome of the study was in-hospital mortality. To develop and validate the final model, we employed the LASSO regression to screen and select relevant variables. Five algorithms, namely Logistic Regression (LR), K-Nearest Neighbors (KNN), Adaptive Boosting (AdaBoost), Random Forest (RF), and eXtreme Gradient Boosting (XGBoost), were utilized. The selection of the best model was based on the area under the curve (AUC) in the validation cohort. Furthermore, we employ the SHapley Additive exPlanations (SHAP) methodology to elucidate the contributions of individual features to the model and analyze their impact on the model’s outputs. To facilitate accessibility, we also created a visual online calculation page for the model. Results In the final cohort comprising 1596 patients from MIMIC-IV and Zhejiang Hospital, 367 individuals (23%) experienced in-hospital mortality during the inpatient follow-up period. After extracting 46 variables from the database, LASSO regression identified 14 predictor variables for further analysis. Among the five evaluated models, the XGBoost model demonstrated superior discriminative power in both the internal validation set (AUC = 0.907) and the external validation set (AUC = 0.787). Furthermore, through the SHAP technique, we identified the top 5 predictors in the feature importance rankings: Glasgow Coma Scale (GCS), Sequential Organ Failure Assessment (SOFA), anticoagulant medication, mannitol medication and oxygen saturation. Conclusions Among the five models, the XGBoost model exhibited superior performance in predicting mortality for patients with sICH in the ICU, indicating its potential significance in the development of early warning systems.
BibTeX
@misc{mao2023machine,
  author = {Baojie Mao and Rui Zhang and Yuhang Pan and Ruzi Zheng and Yanfei Shen and Wei Lü and Yuning Lu and Shanhu Xu and Jiong Wu and Ming Wang and Shu Wan},
  title = {{Machine learning for the prediction of in-hospital mortality in patients with spontaneous intracerebral hemorrhage}},
  year = {2023},
  doi = {10.1101/2023.08.15.23294147},
  howpublished = {medRxiv},
}
Figure