349: AUGMENTING MORTALITY PREDICTION WITH MEDICATION DATA AND MACHINE LEARNING MODELS

Andrea Sikora, Tianyi Zhang, Xianyan Chen, Susan Smith, John W. Devlin, David J. Murphy, Rishikesan Kamaleswaran, Brian MurrayCritical 2025 · DOI

Abstract

Background: In critically ill patients, complex relationships exist among patient disease factors, medication management, and mortality. Considering the potential for nonlinear relationships and the high dimensionality of medication data, machine learning (ML) and advanced regression methods may offer advantages over traditional regression techniques. The purpose of this study was to evaluate the role of different modeling approaches incorporating medication data for mortality prediction.

Methods: This single-center, observational cohort study included 991 adults admitted to the intensive care unit (ICU) for at least 24 hours. Models to predict hospital mortality at discharge were created using 27 predictor variables collected within the first 24 hours of admission, including the Medication Regimen Complexity-ICU (MRC-ICU) score. Models were externally validated against a separate dataset of 4,878 patients. We compared traditional logistic regression, advanced regression (using splines), and supervised ML models (Random Forest, SVM, and XGBoost).

Results: A model including MRC-ICU in addition to SOFA and APACHE II demonstrated an AUROC of 0.83, compared to 0.72 and 0.81 for APACHE II and SOFA alone. ML models demonstrated AUROCs ranging from 0.82 to 0.85. Accuracy of traditional regression models was similar to that of machine learning models.

Conclusions: While medication data were not included as a significant predictor in regression models, the addition of MRC-ICU to severity of illness scores improved mortality prediction. ML methods did not improve model performance relative to traditional regression methods.

Introduction

Predicting mortality in the Intensive Care Unit (ICU) is a cornerstone of critical care research. Standardized tools such as the APACHE II or SOFA scores are widely used to assess illness severity, yet they largely ignore one of the most significant interventions in the ICU: medication. ICU patients typically receive twice as many medications as general ward patients and are highly susceptible to drug-drug interactions and adverse events.

There is a recognized "black box" in current modeling; while clinicians know medications influence outcomes, the high dimensionality and non-linear interactions of these drugs make them difficult to include in traditional statistics. This study investigates whether advanced regression or machine learning can unlock the predictive power of medication data to create more accurate mortality models.

Methodology

Study Design and Data Sources

The researchers conducted a retrospective cohort study using data from the University of North Carolina (UNC) Health System. The primary training and test set consisted of a random sample of 991 adults admitted to the ICU for at least 24 hours between 2015 and 2020. To test the generalizability of the models, an external validation set of 4,878 patients (the "UNC 5000" dataset) was utilized.

Modeling Approaches

Three main modeling categories were compared:

  1. Traditional Logistic Regression: Utilizing linear predictors and stepwise selection by Bayesian information criterion (BIC).
  2. Advanced Logistic Regression: Incorporating natural cubic splines, smoothing splines, and local linearity to account for potential non-linear relationships.
  3. Machine Learning (ML): Supervised classification models including Random Forest, Support Vector Machines (SVM), and XGBoost.

Predictor Variables

The models utilized 27 predictor variables collected within the first 24 hours of ICU admission, including baseline demographics, lab values, and the MRC-ICU score—a validated metric that quantifies the complexity of a patient's medication regimen. Additionally, 14 "missingness indicators" were included to treat the absence of specific data points as a potential predictor itself.

Key Insights and Results

The Value of Medication Complexity

The MRC-ICU score proved to be a valuable addition to clinical models. Adding it to SOFA and APACHE II scores improved the Area Under the Receiver Operating Characteristic (AUROC) from 0.81 and 0.72, respectively, to 0.83. This suggests that the complexity of the prescribed medication regimen serves as a strong proxy for the true severity of a patient's condition.

In the ML models, medication data showed a level of "feature importance" similar to clinical heavyweights like serum lactate levels or ARDS (Acute Respiratory Distress Syndrome) classification, ranking higher than the simple presence of mechanical ventilation.

Machine Learning vs. Traditional Regression

Despite the increasing interest in AI for healthcare, the ML models did not provide a significant accuracy boost over more transparent statistical methods. The SVM model achieved the highest AUROC on the test set at 0.85, but this was closely matched by Nature Cubic Splines Logistic Regression at 0.86. In this context, the "black box" of ML did not outperform interpretable regression techniques.

The Generalization Gap

A significant finding was the drop in performance across all ten models when applied to the validation set. The authors noted structural differences between the training and validation populations—such as a higher proportion of medical ICU admissions in the validation set—highlighting the sensitivity of these models to specific ICU population characteristics.

Discussion and Limitations

While the study successfully demonstrated the utility of the MRC-ICU score, several limitations remain:

  • Sample Size: While robust for clinical studies, the sample sizes may have been relatively small for training highly complex ML models, potentially leading to overfitting.
  • Data Imputation: The use of "normal" values for missing data may have introduced bias.
  • Static Analysis: The study focused only on the first 24 hours of admission. Because critical illness is dynamic, models incorporating data from the entire ICU stay might offer more power.
  • Single-Center Bias: The data originated from a single health system, which may limit generalizability to other regions or hospital types.

Conclusion

Incorporating medication complexity via the MRC-ICU score improves the predictive accuracy of standard mortality models. However, advanced machine learning techniques do not currently offer a clear advantage over traditional regression for this specific task. Future research will focus on larger datasets and high-frequency medication data to develop more granular, causal models of patient outcomes.

Summary updated

BibTeX
@inproceedings{sikora2025augmenting,
  author = {Andrea Sikora and Tianyi Zhang and Xianyan Chen and Susan Smith and John W. Devlin and David J. Murphy and Rishikesan Kamaleswaran and Brian Murray},
  title = {{349: AUGMENTING MORTALITY PREDICTION WITH MEDICATION DATA AND MACHINE LEARNING MODELS}},
  booktitle = {Critical},
  year = {2025},
  doi = {10.1097/01.ccm.0001100060.36305.de},
}

Loading related content...

Figure