A comparison of the ability of the National Early Warning Score and the National Early Warning Score 2 to identify patients at risk of in-hospital mortality: A multi-centre database study

Marco A. F. Pimentel, Oliver Redfern, Stephen Gerry, Gary S. Collins, James Malycha, David Prytherch, Paul E. Schmidt, Gary B. Smith, Peter WatkinsonResuscitation 2018 · DOI

TL;DR

This study compares the effectiveness of the National Early Warning Score (NEWS) and its updated version, NEWS2, in predicting in-hospital mortality for patients with or at risk of Type II Respiratory Failure (T2RF). The findings suggest that NEWS2 does not improve, and in some cases worsens, the predictive ability of NEWS for these specific patient groups, raising questions about its widespread adoption.

What Problem Does This Paper Address?

Early warning scores (EWS) are crucial tools in hospitals for identifying deteriorating patients and triggering timely interventions. The National Early Warning Score (NEWS) is widely used in the UK. However, concerns arose regarding its suitability for patients with Type II Respiratory Failure (T2RF), a condition where targeting normal oxygen saturation levels can be detrimental. NEWS2 was developed with modifications, particularly to its oxygen saturation (SpO2) scoring, to better accommodate these patients. This paper addresses the critical question of whether these NEWS2 modifications actually improve its predictive performance for patients with T2RF or those at risk of it, compared to the original NEWS.

Methods

The researchers conducted a multi-center retrospective observational study using a large database of adult admissions from five UK hospitals. They compared the performance of NEWS and NEWS2 in identifying in-hospital mortality within 24 hours of a vital signs observation. Primary outcomes included in-hospital mortality, while secondary outcomes encompassed unanticipated intensive care unit admission or cardiac arrest within 24 hours. Discrimination was assessed using the c-statistic (area under the receiver operating characteristic curve - AUROC). The study specifically analyzed performance across three groups: patients with documented T2RF, patients at risk of T2RF, and patients not at risk of T2RF.

Datasets

The study utilized a comprehensive database compiled from adult admissions to the Oxford University Hospitals (OUH) group and Portsmouth Hospitals NHS Trust (PH). This database, known as the Hospital Alerting Via Electronic Noticeboard (HAVEN) dataset, included vital signs recorded at the bedside, administrative and demographic information, and data on cardiac arrest, ICU admissions, and hospital discharge status. Prescription data for oxygen was also available for OUH admissions. The study included 251,266 adult admissions in total.

Key Insights and Ideas

  • NEWS2's SpO2 adjustments may not benefit T2RF patients: The core innovation of NEWS2 was its revised SpO2 scoring to better manage patients with T2RF. However, this study found that these changes did not improve NEWS2's predictive power for mortality in this group.
  • NEWS2 may be detrimental for at-risk T2RF patients: Perhaps most surprisingly, NEWS2 showed decreased discrimination for patients identified as "at risk of T2RF" compared to the original NEWS. This suggests that the NEWS2 modifications could lead to missed deteriorations in this vulnerable population.
  • "Error" use of NEWS2 is worse: When NEWS2 was applied to patients not at risk of T2RF (simulating potential misuse), it performed worse than NEWS, indicating a potential for misclassification and inappropriate interventions.
  • No improvement in secondary outcomes: The study also found no performance benefit for NEWS2 over NEWS for secondary outcomes like cardiac arrest or ICU admission.

Key Results

  • Documented T2RF: NEWS2 showed a statistically significant lower discrimination (c-statistic of 0.841) for in-hospital mortality within 24 hours compared to NEWS (0.862).
  • At risk T2RF: NEWS2 also demonstrated lower discrimination (c-statistic of 0.860) than NEWS (0.881) for predicting in-hospital mortality within 24 hours.
  • Not at risk T2RF: When applied to this group, NEWS2 had a lower c-statistic (0.891) compared to NEWS (0.910).
  • Secondary Outcomes: For all secondary endpoints, NEWS2 showed no improvements in discrimination compared to NEWS.

Limitations and Future Work

The authors acknowledge several limitations:

  • Reliance on diagnostic codes and oxygen prescriptions: Categorizing patients as having or being at risk of T2RF relied on these data, which could lead to misclassification. However, they note that COPD coding is generally reliable.
  • Absence of "new confusion" data: NEWS2 incorporates "new confusion" as a vital sign, but this data was not consistently available in their dataset, preventing its inclusion in the analysis.
  • Potential over-representation of patient groups: Analyzing each vital sign set independently might lead to over-representation of certain patient groups.
  • Small number of adverse outcomes: For the "documented T2RF" group, the number of secondary outcomes was less than 100, requiring caution in interpretation.

The authors strongly recommend further evaluation of the relationship between SpO2 values, oxygen therapy, and the risk of adverse outcomes before the wide-scale adoption of NEWS2. They suggest that, in the interim, modifying clinical care escalation protocols and response to triggers might be a more appropriate approach than solely relying on the NEWS2 weighting system.

Summary updated

Abstract

NEWS2 modifications to NEWS do not improve discrimination of adverse outcomes in patients with documented T2RF and decrease discrimination in patients at risk of T2RF. Further evaluation of the relationship between SpO<sub>2</sub> values, oxygen therapy and risk should be investigated further before wide-scale adoption of NEWS2.
BibTeX
@inproceedings{pimentel2018comparison,
  author = {Marco A. F. Pimentel and Oliver Redfern and Stephen Gerry and Gary S. Collins and James Malycha and David Prytherch and Paul E. Schmidt and Gary B. Smith and Peter Watkinson},
  title = {{A comparison of the ability of the National Early Warning Score and the National Early Warning Score 2 to identify patients at risk of in-hospital mortality: A multi-centre database study}},
  booktitle = {Resuscitation},
  year = {2018},
  doi = {10.1016/j.resuscitation.2018.09.026},
}
Figure