Early COVID-19 respiratory risk stratification using machine learning

Molly Douglas, Brian Bell, Adrienne Kinney, Sarah Pungitore, Brian P TonerTrauma 2022 · DOI

TL;DR

Researchers have developed the ECoRRS score, a streamlined machine learning model that predicts whether a COVID-19 patient will require endotracheal intubation within the next 48 hours. Using only six objective clinical variables, the model achieves a 99.7% negative predictive value, offering a reliable way for non-specialists to identify low-risk patients and for hospitals to forecast ventilator demand.

What Problem Does This Paper Address?

During COVID-19 surges, healthcare systems faced a critical triage crisis: how to distinguish between "moderately ill" patients and those on the verge of respiratory failure. This task often fell to clinicians without extensive critical care experience.

While many AI models were rushed to publication early in the pandemic, most were "black boxes" that required too many variables, relied on small datasets, or were too complex for rapid bedside use. The authors sought to create a "sparse" model—one that uses the minimum number of variables possible—to provide a 48-hour "early warning" for intubation using only objective data available at the point of care.

Methods

The researchers compared two different machine learning approaches:

  1. LASSO Regression: A linear model tuned for "sparsity." LASSO penalizes complex models, forcing the algorithm to choose only the most impactful variables. This resulted in the final ECoRRS (Early COVID-19 Respiratory Risk Stratification) score.
  2. XGBoost: A high-performance gradient-boosting algorithm used as a benchmark. While more accurate, it is significantly more complex (using 100 unique decision trees).

The team reformatted Electronic Health Record (EHR) data into 4-hour blocks to match routine nursing checks. They intentionally excluded patient comorbidities and race/ethnicity from the training phase to ensure the model remained robust even when a patient's medical history was unknown or incomplete.

Datasets

  • Source: De-identified data from a hospital-affiliated clinical data warehouse.
  • Scope: 3,447 COVID-19 hospitalizations across three academic medical centers in Arizona between January and April 2020.
  • Prevalence: 20.7% of the cohort required intubation.
  • Split: 80% of the data was used for derivation (training) and 20% for validation (testing).

Key Insights and Ideas

  • The "Sparsity" Trade-off: The authors chose the LASSO-based ECoRRS score over the more accurate XGBoost model. Why? Because ECoRRS only requires six variables and a simple calculator, making it "bedside-usable." This highlights a shift in clinical AI toward interpretability and deployment over raw performance.
  • FiO2 is the Primary Signal: The fraction of inspired oxygen (FiO2) was the most significant predictor. Essentially, the amount of oxygen support a patient already needs is the strongest indicator of their 48-hour trajectory.
  • Optimizing for Safety (NPV): The model was tuned for high sensitivity (90%). This resulted in a Negative Predictive Value (NPV) of 0.997, meaning if the model says a patient is low-risk, clinicians can be extremely confident they won't "crash" in the next two days.
  • Real-World Robustness: The model performed well despite an average missing data rate of 57% in the EHR. This suggests the algorithm is resilient to the "messy" data typical of emergency healthcare environments.

Key Results

  • The ECoRRS Score: The final model uses six variables: FiO2, initial red blood cell count, maximum oxygen saturation (SpO2), lymphocyte count, temperature, and body weight.
  • Predictive Power: The model achieved an Area Under the Curve (AUC) of 0.789 (95% CI 0.785 to 0.812).
  • Triage Utility: At the 90% sensitivity threshold, the model effectively identifies a "safe" subpopulation, allowing hospitals to focus intensive resources on the remaining "at-risk" patients.
  • XGBoost Comparison: While XGBoost reached a higher AUC (0.86), its complexity (requiring specialized software and more inputs) was deemed a barrier to rapid emergency deployment.

Limitations and Future Work

  • Evolving Standards of Care: The data was collected early in the pandemic (Jan–April 2020). Since then, treatments like dexamethasone and "awake proning" have become standard, which may alter the predictive weights of the variables.
  • Subjectivity of the "Ground Truth": The decision to intubate was left to the treating clinician's discretion rather than a strict protocol, introducing potential bias into the training data.
  • Geographic Concentration: The study was limited to Arizona. The authors are currently working on a validation study using a geographically diverse dataset spanning 30 facilities across 6 states to ensure the ECoRRS score holds up in different populations.

Summary updated

Abstract

IV.
BibTeX
@inproceedings{douglas2022early,
  author = {Molly Douglas and Brian Bell and Adrienne Kinney and Sarah Pungitore and Brian P Toner},
  title = {{Early COVID-19 respiratory risk stratification using machine learning}},
  booktitle = {Trauma},
  year = {2022},
  doi = {10.1136/tsaco-2022-000892},
}
Figure