Mortality Prediction in Severe Traumatic Brain Injury Using Traditional and Machine Learning Algorithms

Xiang Wu, Yuyao Sun, Xiao Xu, Ewout W. Steyerberg, Isabel R. A. Retel Helmrich, Fiona Lecky, Jianying Guo, Xiang Li, Junfeng Feng, Qing Mao, Guotong Xie, Andrew I.r. Maas, Guoyi Gao, Jiyao JiangJournal of Neurotrauma 2023 · DOI

TL;DR

This study demonstrates that machine learning, specifically XGBoost, significantly outperforms traditional logistic regression models in predicting in-hospital mortality for severe traumatic brain injury (sTBI) patients. By leveraging a large dataset and a comprehensive set of predictors, XGBoost offers a more accurate and robust tool for clinical decision-making and resource allocation.

What Problem Does This Paper Address?

Predicting the outcome for patients with severe traumatic brain injury (sTBI) is critical for effective clinical management, healthcare policy, and patient/family support. However, the inherent heterogeneity of sTBI, stemming from diverse patient demographics, injury mechanisms, and clinical presentations, makes accurate prognosis challenging. Existing prediction models, often based on traditional statistical methods like logistic regression (LR), have limitations, explaining only a portion of outcome variance and potentially missing complex interactions within the data. This paper aims to develop and validate more accurate prediction models for in-hospital mortality in sTBI patients by exploring the capabilities of machine learning (ML) algorithms.

Methods

The researchers developed and validated several prognostic models for in-hospital mortality in sTBI patients. They employed:

  • Logistic Regression (LR): A standard statistical method for binary outcomes.
  • LASSO Regression: A penalized regression technique that can perform feature selection by shrinking some coefficients to zero, aiming to improve LR's performance and robustness.
  • Machine Learning (ML) Algorithms: Specifically, Support Vector Machines (SVM) and XGBoost (Extreme Gradient Boosting). XGBoost is a powerful ensemble learning method known for its efficiency and performance.

For model development, they used a dataset of 2804 sTBI patients from the CENTER-TBI China Registry and externally validated it on 1113 sTBI patients from the CENTER-TBI European Registry. They included 54 candidate predictors, encompassing demographic, injury-related, clinical severity, radiological, and intervention data. Model performance was evaluated using discrimination (C-statistic) and calibration (intercept and slope). Recursive Feature Elimination (RFE) was used to identify a simplified set of 26 important predictors for XGBoost and SVM. The SHAP (SHapley Additive exPlanations) method was employed to interpret the XGBoost model and identify key predictors.

Datasets

  • Training and Internal Validation Dataset: 2804 patients with severe traumatic brain injury (sTBI) from the Collaborative European NeuroTrauma Effectiveness Research in TBI (CENTER-TBI) China Registry.
  • External Validation Dataset: 1113 patients with sTBI from the CENTER-TBI European Registry.

These datasets share an identical data collection protocol, facilitating robust external validation.

Key Insights and Ideas

  • XGBoost's Superiority: The study strongly suggests that advanced ML algorithms like XGBoost can capture complex, non-linear relationships within sTBI data that traditional LR models miss, leading to significantly better predictive accuracy.
  • The Power of More Predictors (Up to a Point): While LR performance degraded with more predictors, ML models like XGBoost and SVM showed improved discrimination as the number of variables increased, indicating their ability to handle high-dimensional data and extract meaningful signals from noise.
  • Feature Importance Revealed: SHAP analysis provided crucial insights into the most impactful predictors for sTBI mortality, highlighting not only well-known factors like GCS score and age but also emphasizing the importance of less commonly explored variables like emergency interventions and secondary referral.
  • Simplified Models Retain Power: Even with a reduced set of 26 features selected by RFE, the XGBoost and SVM models maintained high performance, suggesting that clinically relevant and parsimonious models can be developed.
  • Web-Based Accessibility: Making the developed models accessible via a web-based calculator is a significant step towards practical clinical implementation and further validation by the research community.

Key Results

  • XGBoost achieved the highest C-statistics:
    • Internal Validation (54 variables): 0.87 (95% CI: 0.81-0.92)
    • External Validation (54 variables): 0.88 (95% CI: 0.87-0.88)
  • Simplified XGBoost (26 variables) also performed exceptionally well:
    • External Validation: 0.87 (95% CI: 0.86-0.87)
  • Outperformed established models: The developed XGBoost model outperformed the widely used CRASH (C-statistic 0.82) and IMPACT (C-statistic 0.80) models in external validation.
  • Key predictors for mortality: Low GCS score, elder age, absent pupillary light reflex, high ISS for brain region, and presence of acute subdural hematoma were the top five predictors.
  • Calibration: ML models (SVM and XGBoost) generally showed better calibration than regression models.

Limitations and Future Work

  • Limited Lab and Radiological Data: The study acknowledges a lack of detailed laboratory indicators and some radiological findings, which could potentially further enhance ML model performance.
  • Fixed Outcome Evaluation: The study used in-hospital mortality as the primary outcome, but a fixed time point for outcome evaluation (e.g., 6 months or 1 year) might provide a more comprehensive picture.
  • Clinical Interventions Specificity: The inclusion of emergency clinical interventions makes the model specific to current practices, and their effectiveness requires further study.
  • Future Directions: The authors suggest that the increasing availability of continuous multi-modality monitoring data in critical care, coupled with advancements in ML, will likely promote the use of ML for TBI outcome prediction. Further studies are needed to provide clinically meaningful decisions and explore the effectiveness of treatments.

Summary updated

Abstract

<b>Abstract</b> Prognostic prediction of traumatic brain injury (TBI) in patients is crucial in clinical decision and health care policy making. This study aimed to develop and validate prediction models for in-hospital mortality after severe traumatic brain injury (sTBI). We developed and validated logistic regression (LR), LASSO regression, and machine learning (ML) algorithms including support vector machines (SVM) and XGBoost models. Fifty-four candidate predictors were included. Model performance was expressed in terms of discrimination (C-statistic) and calibration (intercept and slope). For model development, 2804 patients with sTBI in the Collaborative European NeuroTrauma Effectiveness Research in TBI (CENTER-TBI) China Registry study were included. External validation was performed in 1113 patients with sTBI in the CENTER-TBI European Registry study. XGBoost achieved high discrimination in mortality prediction, and it outperformed logistic and LASSO regression. The XGBoost model established in this study also outperformed prediction models currently available, including the International Mission for Prognosis and Analysis of Clinical Trials (IMPACT) core and International Mission for Prognosis and Analysis of Clinical Trials (CRASH) basic models. When including 54 variables, XGBoost and SVM reached C-statistics of 0.87 (95% confidence interval [CI]: 0.81-0.92) and 0.85 (95% CI: 0.79-0.90) at internal validation, and 0.88 (95% CI: 0.87-0.88) and 0.86 (95% CI: 0.85-0.87) at external validation, respectively. A simplified version of XGBoost and SVM using 26 variables selected by recursive feature elimination (RFE) reached C-statistics of 0.87 (95% CI: 0.82-0.92) and 0.86 (95% CI: 0.80-0.91) at internal validation, and 0.87 (95% CI: 0.87-0.88) and 0.87 (95% CI: 0.86-0.87) at external validation, respectively. However, when the number of variables included decreased, the difference between ML and LR diminished. All the prediction models can be accessed via a web-based calculator. Glasgow Coma Scale (GCS) score, age, pupillary light reflex, Injury Severity Score (ISS) for brain region, and the presence of acute subdural hematoma were the five strongest predictors for mortality prediction. The study showed that ML techniques such as XGBoost may capture information hidden in demographic and clinical predictors of patients with sTBI and yield more precise predictions compared with LR approaches.
BibTeX
@article{wu2023mortality,
  author = {Xiang Wu and Yuyao Sun and Xiao Xu and Ewout W. Steyerberg and Isabel R. A. Retel Helmrich and Fiona Lecky and Jianying Guo and Xiang Li and Junfeng Feng and Qing Mao and Guotong Xie and Andrew I.R. Maas and Guoyi Gao and Jiyao Jiang},
  title = {{Mortality Prediction in Severe Traumatic Brain Injury Using Traditional and Machine Learning Algorithms}},
  journal = {Journal of Neurotrauma},
  year = {2023},
  doi = {10.1089/neu.2022.0221},
}
Figure