TL;DR
This paper introduces a new method for developing "early warning scores" (EWS) that predict patient deterioration using manually recorded vital signs. By analyzing the statistical distribution of these signs, they created an EWS that performs as well as established scores like NEWS, even outperforming EWS derived from continuous monitoring data in some aspects. This offers a more accessible and adaptable way to build robust EWS for hospitals.
What Problem Does This Paper Address?
Hospitals use Early Warning Scores (EWS) to identify patients at risk of sudden deterioration, such as cardiac arrest or needing intensive care. These scores are typically calculated by assigning points to abnormal vital sign readings (like heart rate, blood pressure, respiratory rate, temperature, oxygen saturation). The thresholds for what constitutes an "abnormal" reading are crucial for the EWS's effectiveness.
Previously, the authors developed a centile-based EWS (CEWS) using vital signs automatically collected by bedside monitors. However, many hospitals still rely on manually recorded vital signs, which can differ statistically from continuously monitored data. This discrepancy means EWS developed on one type of data might not perform optimally on the other. This paper addresses this gap by developing and validating a manual CEWS (mCEWS) using manually recorded vital sign data, aiming to create an EWS that is effective in settings where continuous monitoring isn't standard.
Methods
The core of the methodology is an unsupervised, statistical approach to derive vital sign thresholds. Instead of using patient outcomes to define "abnormal," they analyze the statistical distribution of vital signs within a large dataset.
- Centile-Based Thresholds: They used a "development" dataset of manually recorded vital signs to identify the 1st, 5th, 10th, 90th, 95th, and 99th centiles for each vital sign. These centiles then define the boundaries for assigning scores (0, 1, 2, 3) to different ranges of vital sign values. For example, a value falling between the 1st and 5th centile might get a score of 3, indicating a higher risk.
- Kernel Density Estimation: To get a smooth estimate of the vital sign distributions, especially in the tails (extremes), they employed a kernel-based density estimator. This helps in accurately determining the centile values.
- Validation: A separate "validation" dataset was used to test the performance of the newly developed mCEWS. They compared its ability to predict a composite outcome (cardiac arrest, unanticipated ICU admission, or death within 24 hours) against other established EWS, including their own previously developed continuous CEWS (cCEWS).
- Supplemental Oxygen Score: They also investigated the impact of adding a specific score for patients receiving supplemental oxygen, similar to the NEWS system.
Datasets
- Development Dataset: 301,644 vital sign observations from 12,153 admissions (March 2014 - September 2015). This dataset was used to derive the new manual CEWS thresholds.
- Validation Dataset: 1,459,422 vital sign observations from 53,395 admissions (October 2015 - May 2017). This dataset was used to evaluate the performance of the mCEWS and compare it with other EWS.
The data was collected from four Oxford University Hospitals NHS Trust hospitals.
Key Insights and Ideas
- Manual vs. Continuous Data Matters: The statistical distributions of vital signs differ significantly between manually recorded and continuously monitored data. This directly impacts the optimal thresholds for EWS.
- Unsupervised Approach is Powerful: An unsupervised, statistical method for deriving thresholds can create effective EWS without needing explicit patient outcome data during development. This makes the process more reproducible and adaptable.
- Respiratory Rate is Key: Differences in respiratory rate thresholds were identified as a major contributor to the performance gap between the continuous CEWS and the manual CEWS. This highlights the importance of accurately capturing and interpreting respiratory rate.
- Supplemental Oxygen Adds Value: Including a specific score for supplemental oxygen use significantly improved the performance of the manual CEWS, suggesting it's a valuable indicator of patient risk.
Key Results
- The manual CEWS (mCEWS) achieved an Area Under the Receiver Operating Characteristic Curve (AUC) of 0.868 (95% CI, 0.864-0.872) for predicting major adverse events within 24 hours.
- This performance was comparable to the National EWS (NEWS) at 0.867 (0.863-0.871).
- The mCEWS outperformed the original continuous CEWS (cCEWS), which had an AUC of 0.808 (0.804-0.812) on the validation dataset.
- The improvement in mCEWS performance over cCEWS was largely attributed to differences in respiratory rate thresholds.
- The mCEWS without the supplemental oxygen score had an AUC of 0.836, showing the added benefit of this feature.
Limitations and Future Work
- Retrospective Data: The study relied on retrospective analysis of hospitalised patients.
- Repeated Observations: The evaluation used repeated observation sets from the same patient, which, while standard practice, assumes independence that might not hold true in reality.
- Dataset Differences: The development and validation datasets for the continuous CEWS differed from those used for the manual CEWS, including being collected a decade apart and potentially from different hospitals. This could influence results.
- Hypothesized RR Differences: The authors hypothesize that differences in how respiratory rate is measured (manual counting vs. electronic monitoring) explain some of the threshold discrepancies, but other factors cannot be ruled out.
Future Work: * The authors suggest their unsupervised approach can be easily extended to incorporate new variables or develop EWS for specific patient groups (e.g., those with COPD) where current scores may perform less well. * More research is needed to systematically investigate differences between vital signs recorded manually versus by monitors. * Caution is advised when applying EWS designed for manual data to continuously monitored data, and vice versa.
Summary updated