The quality of vital signs measurements and value preferences in electronic medical records varies by hospital, specialty, and patient demographics

Niall Jackson, Jessica E. Woods, Peter Watkinson, Andrew Brent, Tim Peto, A. Sarah Walker, David W. EyreScientific 2023 · DOI

TL;DR

This study reveals that vital sign measurements recorded in electronic health records (EHRs) are not always accurate, exhibiting "value preferences" (e.g., rounding to specific numbers or values). These preferences vary significantly by hospital, patient demographics, and clinical specialty, highlighting a critical issue for patient care and research relying on EHR data.

What Problem Does This Paper Address?

Electronic Health Records (EHRs) are increasingly used for everything from direct patient care to large-scale research. However, the accuracy and reliability of the data within these systems are not always guaranteed. This paper focuses on a specific type of data quality issue: "value preferences" in vital sign measurements. These aren't necessarily outright errors but rather systematic tendencies to record values that are "nicer" or easier to write down, such as rounding to the nearest 10 for blood pressure or recording temperatures as exactly 36.0°C. This phenomenon can introduce bias into clinical decision-making, early warning scores, and research analyses that depend on precise vital sign data. The authors aim to quantify the prevalence of these value preferences and understand what factors influence them.

Methods

The researchers employed a retrospective observational study design using EHR data from Oxford University Hospitals. They developed a statistical model, specifically a maximum likelihood estimator, to quantify the prevalence of value preferences. This model was designed to estimate the underlying true distribution of vital signs while simultaneously accounting for the probability of specific recording preferences. For blood pressure (BP) and heart rate (HR), they modeled rounding to the nearest 10. For respiratory rate (RR), they accounted for rounding to multiples of 2 or 4. A unique aspect of their approach was the specific modeling of an excess of temperature readings at 36.0°C, which they hypothesized might indicate an observation not actually measured but rather assumed or imputed.

To explore the factors associated with these value preferences, they used multivariable logistic regression. This allowed them to investigate the influence of patient demographics (age, sex, ethnicity, deprivation, comorbidities), hospital characteristics (hospital site, specialty), and temporal factors (calendar time, hour of day, days into admission) on the likelihood of observing these value preferences.

Datasets

The study utilized a large dataset of electronic health records from Oxford University Hospitals (OUH) in the UK. This dataset comprised vital sign observations for adult inpatients collected between January 1, 2016, and June 30, 2019. The dataset included over 4.3 million vital sign records from more than 135,000 patients, covering respiratory rate, heart rate, tympanic temperature, systolic and diastolic blood pressure, and oxygen saturations. Additional patient-level data such as age, sex, ethnicity, index of multiple deprivation (IMD), and Charlson comorbidity score were also incorporated.

Key Insights and Ideas

  • "Digit Preference" is Pervasive: The study confirms that systematic rounding and preference for specific values in vital sign recordings are common, not just an occasional anomaly.
  • Temperature's Peculiar Preference: The finding of an unusual excess of 36.0°C temperature readings is particularly striking. This suggests that this value might be used when a temperature isn't actually measured, potentially to represent a "normal" reading without the effort of measurement.
  • Context Matters: Value preferences are not uniformly distributed. They are influenced by a complex interplay of patient characteristics, hospital environment, and even the time of day or length of hospital stay.
  • Implications for AI and Research: The study strongly suggests that any algorithms or research relying on raw vital sign data from EHRs need to account for these systematic biases to avoid flawed conclusions.

Key Results

  • Temperature: An estimated 11.3% of temperature readings were likely recorded as 36.0°C when the true value was different.
  • Blood Pressure (BP) and Heart Rate (HR): Approximately 2.0-2.4% of BP and HR readings were rounded to the nearest 10.
  • Respiratory Rate (RR): Over 22% of RR readings were multiples of 2, and 2.5% were multiples of 4, suggesting a common measurement practice of timing over 30 seconds and doubling.
  • Demographic Associations: Older patients and males were more likely to exhibit value preferences in temperature and BP.
  • Temporal Trends: Value preferences generally decreased over the study period, suggesting potential improvements in data recording practices.
  • Hospital and Specialty Differences: Significant variations were observed across different hospitals and medical specialties, with surgical specialties generally showing higher precision.

Limitations and Future Work

The authors acknowledge several limitations:

  • Single Institution Data: The study is based on data from a single hospital group, limiting generalizability. Future studies are needed to confirm these findings in other settings.
  • Retrospective Nature: The study design prevents identifying the exact reasons behind missing or inaccurate readings (e.g., malfunctioning devices vs. deliberate recording practices).
  • Granularity: The analysis did not account for individual healthcare workers or specific hospital wards, which could introduce further variation.
  • Model Simplifications: The statistical models used are simplifications of complex real-world recording practices. For instance, the temperature model assumes any value not 36.0°C is not subject to digit preference, and the BP/HR models primarily focus on rounding to the nearest 10.
  • Downstream Consequences: The study did not investigate the impact of these value preferences on patient outcomes or the performance of clinical decision support tools.

Future work could involve:

  • Investigating the downstream consequences of value preferences on patient outcomes and clinical decision-making tools.
  • Exploring the practical barriers and behavioral factors contributing to these preferences.
  • Developing more sophisticated statistical models to capture a wider range of recording behaviors.
  • Examining the role of automated measurement systems in reducing value preferences.

Summary updated

Abstract

We aimed to assess the frequency of value preferences in recording of vital signs in electronic healthcare records (EHRs) and associated patient and hospital factors. We used EHR data from Oxford University Hospitals, UK, between 01-January-2016 and 30-June-2019 and a maximum likelihood estimator to determine the prevalence of value preferences in measurements of systolic and diastolic blood pressure (SBP/DBP), heart rate (HR) (readings ending in zero), respiratory rate (multiples of 2 or 4), and temperature (readings of 36.0 °C). We used multivariable logistic regression to investigate associations between value preferences and patient age, sex, ethnicity, deprivation, comorbidities, calendar time, hour of day, days into admission, hospital, day of week and speciality. In 4,375,654 records from 135,173 patients, there was an excess of temperature readings of 36.0 °C above that expected from the underlying distribution that affected 11.3% (95% CI 10.6-12.1%) of measurements, i.e. these observations were likely inappropriately recorded as 36.0 °C instead of the true value. SBP, DBP and HR were rounded to the nearest 10 in 2.2% (1.4-2.8%) and 2.0% (1.3-5.1%) and 2.4% (1.7-3.1%) of measurements. RR was also more commonly recorded as multiples of 2. BP digit preference and an excess of temperature recordings of 36.0 °C were more common in older and male patients, as length of stay increased, following a previous normal set of vital signs and typically more common in medical vs. surgical specialities. Differences were seen between hospitals, however, digit preference reduced over calendar time. Vital signs may not always be accurately documented, and this may vary by patient groups and hospital settings. Allowances and adjustments may be needed in delivering care to patients and in observational analyses and predictive tools using these factors as outcomes or exposures.
BibTeX
@inproceedings{jackson2023quality,
  author = {Niall Jackson and Jessica E. Woods and Peter Watkinson and Andrew Brent and Tim Peto and A. Sarah Walker and David W. Eyre},
  title = {{The quality of vital signs measurements and value preferences in electronic medical records varies by hospital, specialty, and patient demographics}},
  booktitle = {Scientific},
  year = {2023},
  doi = {10.1038/s41598-023-30691-z},
}
Figure