predCOVID-19: A Systematic Study of Clinical Predictive Models for Coronavirus Disease 2019

Patrick Schwab, August DuMont Schütte, Benedikt Dietz, Stefan Bauer

Introduction

During the peak of the COVID-19 pandemic, healthcare systems worldwide faced unprecedented demand for limited resources, including testing kits, hospital beds, and ventilators. Effective resource allocation required a systematic method to stratify patients by risk. This study addresses this gap by developing a machine learning pipeline to predict three critical stages of the COVID-19 clinical pathway: initial diagnosis (testing positive), hospital admission, and the requirement for intensive care (ICU).

The three stages of the COVID-19 clinical pathway modeled in this study: (i) SARS-CoV-2 diagnosis, (ii) hospital admission, and (iii) critical care requirement.

Methodology

The researchers utilized an anonymized dataset of 5,644 patients from the Hospital Israelita Albert Einstein in São Paulo, Brazil, featuring 106 routine clinical, laboratory, and demographic measurements. The study implemented a robust, multistage machine learning pipeline:

  1. Preprocessing: To handle real-world data challenges, features with >99.8% missingness were dropped. Remaining gaps were filled using Multiple Imputation by Chained Equations (MICE). Crucially, "missingness indicators" (binary flags) were added to capture clinical context.
  2. Model Development: Five algorithms were compared: Logistic Regression (LR), Neural Networks (NN), Random Forest (RF), Support Vector Machines (SVM), and Gradient Boosting (XGBoost).
  3. Evaluation: Data was partitioned into training (50%), validation (20%), and test (30%) sets to ensure rigorous performance assessment and prevent overfitting.
  4. Interpretability: The CXPlain (Causal Explanations) framework was used to identify the clinical features driving model predictions.

Key Results

The predictive performance improved significantly as the clinical severity increased:

  • SARS-CoV-2 Test Result: The best model (XGBoost) achieved an Area Under the Curve (AUC) of 0.66. While diagnostic prediction was challenging, it reached a sensitivity of 75%, useful for prioritizing tests.
  • Hospital Admission: The Random Forest model achieved a high accuracy with an AUC of 0.92.
  • Critical Care (ICU) Admission: The SVM model was the top performer, achieving a near-perfect AUC of 0.98. At a fixed sensitivity of 95%, the model maintained a specificity of 95%, indicating a very low false-alarm rate.

Discussion and Insights

A primary finding of the study is that disease severity is significantly easier to predict than the initial diagnosis. This suggests that while viral presence is difficult to detect through routine blood work alone, the body's physiological response—such as inflammation and organ stress—leaves a distinct signature that models can identify with high precision.

Feature importance across the three prediction tasks. Note the high marginal contribution of "missingness" indicators, particularly for the initial test result.

Furthermore, the study revealed that "missingness" is a highly informative feature. For example, the absence of an Arterial Lactic Acid test was a top predictor for a positive COVID-19 result. This likely reflects the implicit clinical judgment of physicians, who only order specific, invasive tests when they suspect a patient is severely ill. Finally, the results demonstrate that routine clinical data—such as lactate dehydrogenase, glucose, and pH levels—is sufficient for high-accuracy triage without the need for expensive genomic or imaging modalities.

Limitations

While promising, the study is limited by its single-site nature, as clinical protocols in Brazil may differ from other regions. Additionally, the reliance on missingness indicators introduces a "doctor-in-the-loop" bias, where the model reflects existing clinical heuristics rather than pure physiological signals. Future work could involve validating these models on more diverse global cohorts and integrating additional data modalities like chest X-rays.

Summary updated

Abstract

Coronavirus Disease 2019 (COVID-19) is a rapidly emerging respiratory disease caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Due to the rapid human-to-human transmission of SARS-CoV-2, many healthcare systems are at risk of exceeding their healthcare capacities, in particular in terms of SARS-CoV-2 tests, hospital and intensive care unit (ICU) beds and mechanical ventilators. Predictive algorithms could potentially ease the strain on healthcare systems by identifying those who are most likely to receive a positive SARS-CoV-2 test, be hospitalised or admitted to the ICU. Here, we study clinical predictive models that estimate, using machine learning and based on routinely collected clinical data, which patients are likely to receive a positive SARS-CoV-2 test, require hospitalisation or intensive care. To evaluate the predictive performance of our models, we perform a retrospective evaluation on clinical and blood analysis data from a cohort of 5644 patients. Our experimental results indicate that our predictive models identify (i) patients that test positive for SARS-CoV-2 a priori at a sensitivity of 75% (95% CI: 67%, 81%) and a specificity of 49% (95% CI: 46%, 51%), (ii) SARS-CoV-2 positive patients that require hospitalisation with 0.92 AUC (95% CI: 0.81, 0.98), and (iii) SARS-CoV-2 positive patients that require critical care with 0.98 AUC (95% CI: 0.95, 1.00). In addition, we determine which clinical features are predictive to what degree for each of the aforementioned clinical tasks. Our results indicate that predictive models trained on routinely collected clinical data could be used to predict clinical pathways for COVID-19, and therefore help inform care and prioritise resources.
BibTeX
@misc{schwab2020predcovid,
  author = {Patrick Schwab and August DuMont Schütte and Benedikt Dietz and Stefan Bauer},
  title = {{predCOVID-19: A Systematic Study of Clinical Predictive Models for Coronavirus Disease 2019}},
  year = {2020},
  eprint = {2005.08302},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2005.08302},
}
Figure