Early prediction of level-of-care requirements in patients with COVID-19

Boran Hao, Shahabeddin Sotudian, Taiyao Wang, Tingting Xu, Yang Hu, Apostolos Gaitanidis, Kerry Breen, George C. Velmahos, Ioannis Ch. PaschalidiseLife 2020 · DOI

TL;DR

This paper presents machine learning models that can accurately predict the need for hospitalization, ICU admission, and mechanical ventilation in COVID-19 patients using readily available clinical and laboratory data. These models achieve high accuracy (up to 88%) and significantly outperform traditional pneumonia severity scores, offering a valuable tool for clinical decision-making and resource allocation during pandemics.

What Problem Does This Paper Address?

The COVID-19 pandemic placed immense strain on healthcare systems worldwide, necessitating rapid and accurate assessment of patient severity to manage limited resources like ICU beds and ventilators. While models for predicting disease spread existed, there was a critical need for individual-level prediction of patient outcomes to guide clinical triage and care. Existing methods for assessing pneumonia severity were not specifically tailored to COVID-19 and showed limited predictive power in this context. This paper aims to bridge this gap by developing and validating robust predictive models for COVID-19 patient outcomes.

Methods

The researchers employed a multi-stage approach to develop their predictive models:

  • Data Extraction: Natural Language Processing (NLP) techniques were used to extract a wide range of features from electronic health records, including demographics, vital signs, symptoms, pre-existing conditions, medications, and radiology reports.
  • Feature Engineering & Selection: Categorical features were one-hot encoded, and missing values were imputed. Extensive feature selection methods, including statistical tests and recursive feature elimination, were used to identify the most predictive variables for each outcome.
  • Classification Models: Both nonlinear ensemble methods (Random Forests, XGBoost) and interpretable linear models (Support Vector Machines with L1/L2 regularization, Logistic Regression) were explored. Parsimonious models with a limited number of features were specifically developed for ease of use and interpretability.
  • Model Validation: Models were validated using two strategies: random splitting of data into training and testing sets, and training on a subset of hospitals and testing on a separate hospital. Performance was evaluated using Area Under the Curve (AUC) and Weighted-F1 score.

Datasets

The study utilized a dataset of 2566 consecutive COVID-19 patients who tested positive for SARS-CoV-2 between March 4 and April 13, 2020, from five Massachusetts hospitals within the same healthcare system: Massachusetts General Hospital (MGH), Brigham and Women's Hospital (BWH), Faulkner Hospital (FH), Newton-Wellesley Hospital (NWH), and North Shore Medical Center (NSM).

Key Insights and Ideas

  • Clinical Data is Highly Predictive: Even with a relatively small set of key features, accurate predictions of hospitalization, ICU admission, and mechanical ventilation are achievable upon a patient's initial presentation. This suggests a distinct "signature" for severe COVID-19 disease.
  • Parsimonious Models are Effective: Simpler, interpretable models with fewer variables performed nearly as well as more complex models, making them more practical for clinical implementation.
  • Traditional Scores Fall Short: Standard pneumonia severity scores like PSI and CURB-65 demonstrated significantly lower predictive accuracy for COVID-19 outcomes compared to the developed models.
  • Predictive Power Extends into the Future: The models showed the ability to predict ICU admission and ventilation needs several hours in advance, offering valuable lead time for clinical intervention.
  • ACE Inhibitors May Be Protective: The hospitalization model suggested that ACE inhibitors might slightly reduce the odds of hospitalization, contrary to some initial speculations.

Key Results

  • Hospitalization Prediction: Achieved a validation accuracy of up to 88% using clinical and laboratory data. Key predictors included vital signs, age, BMI, and comorbidities.
  • ICU Admission Prediction: Achieved a validation accuracy of up to 87%. Important predictors included chest imaging opacities, age, vital signs, gender, laboratory results, and diabetes.
  • Mechanical Ventilation Prediction: Achieved a validation accuracy of up to 86%. Similar predictors to ICU admission were identified.
  • Comparison to Standard Scores: The developed models outperformed PSI and CURB-65 scores, with AUCs for ICU and ventilation prediction reaching up to 86% and 73%/67% respectively for the new models versus standard scores.
  • Restricted Models: Even when focusing on more complex cases (hospitalized patients with available lab data), ICU and ventilation prediction models maintained high accuracy (83% and 82% respectively).

Limitations and Future Work

  • Data Specificity: The models were trained on data from a specific time period and geographic location (Massachusetts, USA, early in the pandemic). Their generalizability to different populations, variants, or healthcare settings may require further validation.
  • Data Availability: While the study aimed to use readily available data, some crucial lab values (like LDH and CRP) were not available for all patients, necessitating the development of restricted models.
  • Causality vs. Correlation: The study identifies strong predictors, but establishing definitive causal relationships requires further investigation.
  • Future Work: The authors suggest using these models to guide patient triage, optimize staffing and resource planning, and inform future policy decisions, such as vaccination prioritization. They also highlight the potential for developing calculators based on these parsimonious models for clinical use.

Summary updated

Abstract

This study examined records of 2566 consecutive COVID-19 patients at five Massachusetts hospitals and sought to predict level-of-care requirements based on clinical and laboratory data. Several classification methods were applied and compared against standard pneumonia severity scores. The need for hospitalization, ICU care, and mechanical ventilation were predicted with a validation accuracy of 88%, 87%, and 86%, respectively. Pneumonia severity scores achieve respective accuracies of 73% and 74% for ICU care and ventilation. When predictions are limited to patients with more complex disease, the accuracy of the ICU and ventilation prediction models achieved accuracy of 83% and 82%, respectively. Vital signs, age, BMI, dyspnea, and comorbidities were the most important predictors of hospitalization. Opacities on chest imaging, age, admission vital signs and symptoms, male gender, admission laboratory results, and diabetes were the most important risk factors for ICU admission and mechanical ventilation. The factors identified collectively form a signature of the novel COVID-19 disease.
BibTeX
@inproceedings{hao2020early,
  author = {Boran Hao and Shahabeddin Sotudian and Taiyao Wang and Tingting Xu and Yang Hu and Apostolos Gaitanidis and Kerry Breen and George C. Velmahos and Ioannis Ch. Paschalidis},
  title = {{Early prediction of level-of-care requirements in patients with COVID-19}},
  booktitle = {eLife},
  year = {2020},
  doi = {10.7554/elife.60519},
}
Figure