Machine-Learning Enhanced Prediction of Need for Hemorrhage Resuscitation after Trauma – The ShockMatrix Pilot Study

Tobias Gauss, Jean-Denis Moyer, Clélia Colas, Manuel Pichon, Nathalie Delhaye, Marie Werner, Véronique Ramonda, Theophile Sempe, Sofiane Medjkoune, Julie Josse, Arthur James, Anatole Harrois, The Traumabase GroupResearch Square 2024 · DOI

TL;DR

The ShockMatrix study introduces a machine-learning pipeline designed to predict the "Need for Hemorrhage Resuscitation" (NHR) using only prehospital data. By combining a high-performing XGBoost model with a user-friendly smartphone app, the researchers demonstrated that AI can provide early, actionable decision support in the high-pressure environment of trauma centers without disrupting clinical workflows.

What Problem Does This Paper Address?

In major trauma, every minute counts. Deciding whether a patient needs immediate, aggressive hemorrhage resuscitation (like blood transfusions or emergency surgery) is a high-stakes task. Currently, clinicians often rely on "gestalt" (clinical intuition) or simple scoring systems. However, human decision-making under stress can be inconsistent, and many existing predictive models rely on hospital-based lab results that arrive too late to be truly "early" warnings.

The authors identify a critical gap: the lack of tools that are both predictive (using only early, prehospital data) and practical (integrated into the actual workflow of a trauma leader). The ShockMatrix study aims to bridge this "implementation gap" by developing a model and testing its real-world feasibility via a mobile application.

Methods

The study was conducted in two distinct parts:

  1. Model Development: The team evaluated four tree-based algorithms (CART, Random Forest, XGBoost, and CatBoost) using a massive registry of trauma data.
  2. Feature Selection: They used SHAP (Shapley Additive exPlanations) values to identify the nine most impactful prehospital variables, such as systolic blood pressure, heart rate, and capillary hemoglobin.
  3. Optimization: Because missing a bleeding patient is far worse than a "false alarm," they optimized the models using the F4-score, which heavily weights sensitivity (recall) over precision.
  4. Calibration: They applied Bayes Minimum Risk theory to recalibrate the model's probabilities, ensuring the predicted risks aligned with real-world frequencies.
  5. The Pilot Study: They developed a HIPAA-compliant smartphone app (hosted on Microsoft Azure). During a three-month period, trauma leaders at five centers used the app to input the nine prehospital variables during the "prealert" phase—the window of time after a call comes in but before the patient arrives.

Datasets

  • Traumabase Registry: A high-quality French registry. The authors used data from 28,614 patients (collected between 2010 and 2022) for the initial model training, validation, and testing.
  • Pilot Cohort: A new, prospective dataset of 139 patients was used to evaluate the app's usability and the model's real-time feasibility.

Key Insights and Ideas

  • The "Prealert" Sweet Spot: The study identifies the prealert phase as the ideal time for data entry. By capturing data then, the model can generate a prediction exactly when the trauma team is preparing their resources.
  • Prioritizing Sensitivity: By using the F4-score, the researchers explicitly designed the AI to be "cautious." In trauma, a false positive means you prepared blood that wasn't used; a false negative could mean a preventable death.
  • Explainability as a Feature: Using SHAP values wasn't just for the researchers; it ensures the model's "logic" (e.g., "this patient is high risk because their blood pressure is low and they are intubated") aligns with clinical reasoning, which is vital for physician trust.
  • Workflow over Raw Power: The study emphasizes that a slightly less accurate model that is actually used is more valuable than a "perfect" model that requires data clinicians don't have time to enter.

Key Results

  • Model Performance: The XGBoost model achieved an AUC-ROC of 0.89 and an F4-score of 0.76, outperforming established clinical scores like the ABC score and "Red Flag" criteria.
  • Speed and Usability: In the pilot, the median time to input data into the smartphone app was less than 2 minutes.
  • Clinician Acceptance: 87% of participating clinicians reported global satisfaction with the tool, and 91% found the app ergonomic and user-friendly.
  • Clinical Parity: The model’s Likelihood Ratios (LR+ 3.44) were comparable to the "gestalt" of experienced trauma leaders (LR+ 3.1), suggesting the AI can serve as a reliable "second opinion" or an objective benchmark.

Limitations and Future Work

  • Internal Validation: While the model was tested on a "held-out" portion of the Traumabase, it has not yet been validated on a completely independent, external dataset from a different country or registry.
  • Pilot Scale: The pilot study was a feasibility test; it was not large enough to statistically prove that the model improves patient outcomes (like mortality).
  • The "Black Box" Barrier: In this pilot, clinicians were blinded to the model's prediction to minimize risk. The next logical step—and a significant hurdle—is a "closed-loop" study where clinicians actually see and act on the AI's output.
  • Future Directions: The authors are currently moving toward a larger prospective study across 16 dispatch centers to further validate the tool's predictive power in diverse settings.

Summary updated

Abstract

<title>Abstract</title> Importance: Decision-making in trauma patients remains challenging and often result in deviation from guidelines. Machine-Learning (ML) enhanced decision-support could improve hemorrhage resuscitation. Aim To develop a ML enhanced decision support tool to predict <italic>Need for Hemorrhage Resuscitation (NHR)</italic> (part I) and test the collection of the predictor variables in real time in a smartphone app (part II). Design, Setting, and Participants: Development of a ML model from a registry to predict <italic>NHR</italic> relying exclusively on prehospital predictors. Several models and imputation techniques were tested. Assess the feasibility to collect the predictors of the model in a customized smartphone app during prealert and generate a prediction in four level-1 trauma centers to compare the predictions to the <italic>gestalt</italic> of the trauma leader. Main Outcomes and Measures: Part 1: Model output was <italic>NHR</italic> defined by 1) at least one RBC transfusion in resuscitation, 2) transfusion ≥ 4 RBC within 6 hours, 3) any hemorrhage control procedure within 6 hours or 4) death from hemorrhage within 24 hours. The performance metric was the F4-score and compared to reference scores (<italic>RED FLAG, ABC</italic>). In part 2, the model and clinician prediction were compared with Likelihood Ratios (LR). Results From 36325 eligible patients in the registry (Nov 2010 - May 2022), 28614 were included in the model development (Part 1). Median age was 36 [25–52], median ISS 13 [5–22], 3249/28614 (11%) corresponded to the definition of <italic>NHR</italic>. A XGBoost model with nine prehospital variables generated the best predictive performance for <italic>NHR</italic> according to the F4-score with a score of 0.76 [0.73–0.78]. Over a 3-month period (Aug - Oct 2022), 139 of 391 eligible patients were included in part II (38.5%), 22/139 with <italic>NHR</italic>. Clinician satisfaction was high, no workflow disruption observed and LRs comparable between the model and the clinicians. Conclusions and Relevance: The ShockMatrix pilot study developed a simple ML-enhanced <italic>NHR</italic> prediction tool demonstrating a comparable performance to clinical reference scores and clinicians. Collecting the predictor variables in real-time on prealert was feasible and caused no workflow disruption.
BibTeX
@inproceedings{gauss2024machine,
  author = {Tobias Gauss and Jean-Denis Moyer and Clélia Colas and Manuel Pichon and Nathalie Delhaye and Marie Werner and Véronique Ramonda and Theophile Sempe and Sofiane Medjkoune and Julie Josse and Arthur James and Anatole Harrois and The Traumabase Group},
  title = {{Machine-Learning Enhanced Prediction of Need for Hemorrhage Resuscitation after Trauma – The ShockMatrix Pilot Study}},
  booktitle = {Research Square},
  year = {2024},
  doi = {10.21203/rs.3.rs-3928257/v1},
}
Figure