Development and Validation of a Two-Step Predictive Risk Stratification Model for Coronavirus Disease 2019 In-hospital Mortality: A Multicenter Retrospective Cohort Study

Yang Li, Yanlei Kong, Mark H. Ebell, Leonardo Martínez, Xinyan Cai, Robert P. Lennon, Derjung M. Tarn, Arch G. Mainous, Aleksandra Zgierska, Bruce Barrett, Wen‐Jan Tuan, Kevin Maloy, Munish Goyal, Alex H. Krist, Tamas S. Gal, Meng-Hsuan Sung, Changwei Li, Yier Jin, Ye ShenFrontiers in Medicine 2022 · DOI

TL;DR

This paper introduces a novel two-step risk stratification model to predict in-hospital mortality for COVID-19 patients. By prioritizing easily obtainable data in the first step, it efficiently identifies low- and high-risk individuals, reserving more complex testing for those in an intermediate risk category, thereby optimizing resource allocation and patient care.

What Problem Does This Paper Address?

The COVID-19 pandemic placed immense strain on healthcare systems, particularly concerning hospital bed availability and the need for intensive care. Clinicians require accurate, rapid, and cost-effective tools to triage patients, identifying those most likely to benefit from hospitalization and intensive support. Existing prognostic scores often rely on extensive laboratory tests or imaging, which can be time-consuming, expensive, and burdensome for already stretched healthcare resources. Furthermore, models developed in one region may not perform well in others due to population heterogeneity. This study aims to bridge this gap by developing and validating a two-step predictive model that leverages readily available data for initial triage, with a secondary step incorporating laboratory data for a more refined assessment.

Methods

The researchers developed a two-step predictive risk stratification model.

  • Step 1: This initial step focuses on identifying patients at clearly low or high risk using easily collected, routinely available data. This includes demographic information (age) and vital signs (respiratory rate, SpO2, heart rate), along with the number of pre-existing comorbidities. A multivariable logistic model with a stepwise variable selection procedure was employed, followed by rounding of coefficients to create a simple point score. Patients scoring below a certain threshold were classified as low-risk, and those above another threshold as high-risk.
  • Step 2: Patients not classified in Step 1 proceed to Step 2. This step incorporates additional, more difficult-to-obtain laboratory variables (blood urea nitrogen, neutrophil count, C-reactive protein, platelets, neutrophil-to-lymphocyte ratio) alongside the variables from Step 1. A similar scoring procedure was used to further stratify these remaining patients into moderate and higher risk groups.

The model development followed TRIPOD principles, and multiple imputation was used to handle missing data.

Datasets

The study utilized retrospective data from four health centers in the United States:

  • Virginia Commonwealth University (VCU) - used as the derivation cohort.
  • Georgetown University (GU)
  • University of Florida (UFL)
  • University of California, Los Angeles (UCLA) - these three were used as validation cohorts.

A total of 1,673 participants from VCU were included in the derivation cohort. The validation cohorts comprised 558 (GU), 1,815 (UFL), and 1,570 (UCLA) individuals. The data collection periods varied across the sites, ranging from March 2020 to June 2021 for VCU.

Key Insights and Ideas

  • Efficiency through staged assessment: The core innovation is the two-step approach. By using readily available data first, the model quickly categorizes a significant portion of patients, saving time and resources. Only a subset of patients requires the more resource-intensive second step.
  • "Point-of-care" data is powerful: The first step demonstrates that basic clinical information, often available at the bedside, is highly predictive of COVID-19 mortality. This makes the model practical for immediate clinical use.
  • Addressing heterogeneity: Validating the model across multiple diverse health systems in the US was crucial. While performance varied slightly, the model showed consistent utility, suggesting its potential for broader application.
  • Improved discrimination over one-step models: The two-step model consistently outperformed a traditional one-step model in terms of discrimination (C-statistics) and calibration, highlighting the benefit of its staged approach.

Key Results

  • The two-step model achieved good discrimination with internal validation at VCU (C-statistic: 0.83, 95% CI 0.79-0.88) and external validation across the other health systems (range: 0.79-0.85).
  • In the derivation cohort, the first step successfully classified patients into lower-risk (2.0% mortality) and higher-risk (30.1% mortality) groups.
  • The overall mortality rates across the combined risk groups (lower, moderate, higher) were 1.9%, 7.6%, and 33.3%, respectively, showing good separation and a clear trend.
  • The two-step model demonstrated better net benefits in decision curve analysis compared to a direct one-step method, indicating greater clinical utility.
  • The two-step method classified significantly more individuals into the lower- or higher-risk groups in the first step compared to a one-step method, suggesting it can triage a larger proportion of patients more efficiently.

Limitations and Future Work

  • Data limitations: The study acknowledges that variant information for SARS-CoV-2 was not available, and the majority of patients were likely infected with the wild-type. Future validation with emerging variants is warranted.
  • Vaccination status: Data on vaccination status was not collected, meaning the findings primarily apply to unvaccinated populations. The impact of vaccination on risk stratification needs further investigation.
  • Timing of admission: The exact timing of patient admission relative to symptom onset was not captured, which could influence prediction accuracy. Incorporating days from initial symptom onset might improve the model.
  • Generalizability: While validated across multiple US sites, further prospective, regional studies are needed to precisely assess heterogeneity bias and refine the model for diverse populations.
  • Future development: The authors suggest further validation and potential development of new triage scores that incorporate newer variants and vaccination status.

Summary updated

Abstract

The two-step scoring system used widely available, point-of-care data for triage of COVID-19 patients and is a potentially time- and cost-saving tool in practice.
BibTeX
@inproceedings{li2022development,
  author = {Yang Li and Yanlei Kong and Mark H. Ebell and Leonardo Martínez and Xinyan Cai and Robert P. Lennon and Derjung M. Tarn and Arch G. Mainous and Aleksandra Zgierska and Bruce Barrett and Wen‐Jan Tuan and Kevin Maloy and Munish Goyal and Alex H. Krist and Tamas S. Gal and Meng-Hsuan Sung and Changwei Li and Yier Jin and Ye Shen},
  title = {{Development and Validation of a Two-Step Predictive Risk Stratification Model for Coronavirus Disease 2019 In-hospital Mortality: A Multicenter Retrospective Cohort Study}},
  booktitle = {Frontiers in Medicine},
  year = {2022},
  doi = {10.3389/fmed.2022.827261},
}
Figure