Risk Prediction for Poor Outcome and Death in Hospital In-Patients with COVID-19: Derivation in Wuhan, China and External Validation in London, UK

Huayu Zhang, Ting Shi, Xiaodong Wu, Xin Zhang, Kun Wang, Daniel Bean, Richard Dobson, James Teo, Jiaxing Sun, Pei Zhao, Chenghong Li, Kevin Dhaliwal, Honghan Wu, Qiang Li, Bruce GuthrieSSRN 2020 · DOI

Introduction

During the height of the COVID-19 pandemic, healthcare systems worldwide faced unprecedented pressure. Clinicians required rapid, objective tools to assist in triage—specifically to identify which patients required immediate hospitalization or intensive care and which could be safely monitored at home. While many prognostic models were developed early in the pandemic, their generalizability across different healthcare systems remained a significant concern. This study addresses this "generalizability gap" by developing a risk prediction model in Wuhan, China, and performing external validation on a cohort in London, UK.

Methodology

The researchers utilized multivariable logistic regression to construct their predictive models. To ensure the models captured generalizable patterns rather than dataset-specific noise (overfitting), they applied L1-regularization (LASSO).

Three distinct model tiers were developed based on the availability of clinical data:

  1. DCS: Demographics, Premorbid Conditions, and Symptoms (intended for community-level triage).
  2. DCSL: The full model including Laboratory results.
  3. DL: A simplified hospital-focused model using only Demographics and Laboratory results.

The "DL" model emerged as the most robust and practical for clinical use. It relies on seven key predictors: age, sex, and five routine laboratory markers: neutrophil count, lymphocyte count, platelet count, C-reactive protein (CRP), and creatinine.

Data Cohorts and the "Casemix" Challenge

A critical aspect of this study is the stark difference between the derivation and validation cohorts, which highlights the difficulty of cross-border medical AI:

  • Wuhan, China (Derivation): 775 adult inpatients from Wuhan Sixth Hospital and Taikang Tongji Hospital. This group had a relatively low mortality rate of 4.3%.
  • London, UK (External Validation): 226 adult inpatients from King’s College Hospital. This group was significantly older and sicker, with a mortality rate of 34.1%.

This eight-fold difference in mortality illustrates the "casemix" challenge. The London cohort represented a much higher-risk population than the one used to build the model. This "spectrum effect" means that a model calibrated for a lower-risk population will likely under-predict risk when applied to a higher-risk setting.

Key Results and Performance

In its original environment in Wuhan, the DL-Death model demonstrated excellent discrimination with a C-index of 0.91 (where 1.0 is perfect). However, when applied to the London cohort, performance dropped to a C-index of 0.74 for death and 0.72 for poor outcomes.

While a C-index of 0.74 is still considered "good" in clinical practice, the decline underscores the risks of deploying medical AI without local validation. Furthermore, the definition of "low risk" shifted dramatically between locations. In Wuhan, the model-identified low-risk group (88.8% of patients) had a death rate of only 0.34%. In London, the group identified as low-risk by the same parameters still faced a 16.5% mortality rate.

Discussion and Limitations

The study found that once a patient is hospitalized, simple laboratory biomarkers (like CRP and neutrophil counts) are significantly more predictive of outcomes than reported symptoms or underlying comorbidities. In adjusted models, most symptoms lost their statistical significance compared to these lab markers.

However, the study faced several limitations:

  • Missing Data: The retrospective nature of the study meant that some key predictors, such as oxygen saturation ($SpO_2$), were not available in the datasets despite being known predictors of poor outcomes.
  • Sample Size: While larger than many early studies, the validation group was relatively small, which can affect statistical certainty.
  • Community Triage Gap: Most models are built on hospitalized patients, yet the most critical triage decisions often occur before a patient is admitted to a ward.

Conclusion

The findings emphasize that there is no "one-size-fits-all" model for a global pandemic response. Because patient demographics, testing protocols, and healthcare systems differ so widely, models must be locally validated and recalibrated. The authors conclude that the medical community needs collaborative, federated datasets to allow for the rapid, real-time validation of AI tools to suit specific clinical environments.

Summary updated

BibTeX
@misc{zhang2020risk,
  author = {Huayu Zhang and Ting Shi and Xiaodong Wu and Xin Zhang and Kun Wang and Daniel Bean and Richard Dobson and James Teo and Jiaxing Sun and Pei Zhao and Chenghong Li and Kevin Dhaliwal and Honghan Wu and Qiang Li and Bruce Guthrie},
  title = {{Risk Prediction for Poor Outcome and Death in Hospital In-Patients with COVID-19: Derivation in Wuhan, China and External Validation in London, UK}},
  year = {2020},
  doi = {10.2139/ssrn.3590468},
  howpublished = {SSRN},
}
Figure