The quality of vital signs measurements in electronic medical records varies by hospital, specialty, and patient demographics

Niall Jackson, Jessica E. Woods, Peter Watkinson, Andrew Brent, Tim Peto, A. Sarah Walker, David W. EyremedRxiv 2022 · DOI

TL;DR

This study reveals significant "digit preference"—the tendency to round numbers—in vital signs recorded within Electronic Health Records (EHRs). Analyzing over 4 million records, researchers found that temperature and blood pressure readings are frequently rounded to "clean" values (like 36.0°C or multiples of ten), with the frequency of rounding influenced by patient age, sex, hospital specialty, and even the time of day.

What Problem Does This Paper Address?

As healthcare increasingly relies on Big Data, EHRs have become the backbone of clinical research, machine learning diagnostics, and hospital quality improvement. However, most data quality research focuses on "coding" errors (e.g., mislabeling a diagnosis). This paper tackles a more subtle but pervasive issue: the accuracy of the raw numeric values themselves.

The authors investigate digit preference, a psychological bias where humans recording data gravitate toward "round" or "normal" numbers. If a patient’s temperature is 36.1°C, a busy clinician might simply record 36.0°C. While seemingly minor, this systematic bias can distort epidemiological models, trigger false clinical alarms, or mask subtle physiological trends in large-scale observational studies.

Methods

The researchers conducted a retrospective observational study using multivariable logistic regression to identify factors associated with digit preference.

A key technical detail is their use of natural cubic splines to model non-linear relationships between continuous variables (like age or time) and the probability of a rounded recording. They specifically looked for:

  • Temperature: Readings of exactly 36.0°C (the lower bound of "normal").
  • Blood Pressure (BP): Both systolic and diastolic readings ending in zero.

The study is particularly interesting because the hospitals used a semi-automated data capture system. Vital signs were measured by a machine but then manually transcribed into a tablet by a human. This "human-in-the-loop" stage is the primary suspect for the introduced bias.

Datasets

The study utilized a massive dataset from the Oxford University Hospitals (OUH) NHS Foundation Trust in the UK.

  • Scope: Data from four teaching hospitals between January 2016 and June 2019.
  • Volume: Initially 5.8 million sets of vital signs, narrowed down to 4,305,914 complete records from 143,352 unique patients.
  • Variables: Included respiratory rate, heart rate, temperature, and blood pressure, alongside patient demographics (age, sex, ethnicity, deprivation) and hospital factors (specialty, ward, time of admission).

Key Insights and Ideas

  • The "Normalcy" Bias: Digit preference was significantly more likely to occur if the previous reading for that patient was normal. If a patient is stable, clinicians appear more likely to "autopilot" the data entry. Conversely, abnormal readings led to much higher precision in subsequent measurements.
  • Surgical vs. Medical Precision: Surgical specialties generally recorded vital signs with higher precision than medical specialties. This might reflect different ward cultures or the perceived "criticality" of specific metrics in different contexts (e.g., temperature in oncology vs. orthopedics).
  • The Admission Effect: Data precision was highest at the time of hospital admission. As a patient’s stay lengthened, the frequency of rounded numbers increased, suggesting a drop in recording rigor as the patient becomes a "familiar" face on the ward.
  • Demographic Skew: Rounding was more common in older and male patients. While the reasons aren't fully clear, the authors suggest physical barriers (like patient agitation) or perceived acuity might play a role.

Key Results

  • Temperature Inflation: 15.1% of all temperature readings were recorded as 36.0°C. Based on the underlying distribution, only 4.9% were expected to be that value. This means roughly 10% of all temperature records were likely rounded.
  • Blood Pressure Rounding: 2.2% of BP readings had both numbers ending in zero, double the 1% expected by chance.
  • Temporal Trends: Digit preference actually decreased over the three-year study period, suggesting that institutional data hygiene can improve over time, even without a specific intervention.
  • Circadian Patterns: Temperature was most likely to be recorded accurately in the afternoon/evening, while BP rounding peaked in the evening.

Limitations and Future Work

The primary limitation is that the study is based on a single hospital group using a specific semi-automated entry system. Results might differ in hospitals using fully automated "plug-and-play" sensors that upload data directly to the EHR without human transcription.

Additionally, the researchers could not identify individual staff members, meaning they couldn't determine if digit preference is a universal human tendency or driven by a small subset of "heavy rounders."

Future Work: The authors call for research into the "downstream consequences" of this bias—specifically, whether rounded data leads to different clinical outcomes or if researchers need to build "uncertainty buffers" into their observational models to account for this human-induced noise.

Summary updated

Abstract

Abstract Objective To assess the frequency of digit preference in recording of vital signs in electronic healthcare records (EHRs) and associated patient and hospital factors. Study Design and Setting We used EHR data from Oxford University Hospitals, UK, between 01-January-2016 and 30-June-2019 and multivariable logistic regression to investigate associations between temperature readings of 36.0°C or systolic and diastolic blood pressure (SBP/DBP) readings both ending in zero and patient age, sex, ethnicity, deprivation, comorbidities, calendar time, hour of day, days into admission, hospital, day of week and speciality. Results In 4,305,914 records from 143,352 patients, there was an excess of temperature readings of 36.0°C (15.1%, 649,976/4,305,914), compared to an expected 4.9% from the underlying distribution. 2.2% (95,215) BP readings had a SBP and DBP both ending in zero vs. 1% expected by chance. Digit preference was more common in older and male patients, as length of stay increased, following a previous normal set of vital signs and typically more common in medical vs. surgical specialities. Differences were seen between hospitals, however, digit preference reduced over calendar time. Conclusion Vital signs may not always be accurately documented. Allowances and adjustments may be needed in observational analyses using these factors as outcomes or exposures. What’s New? Key findings ▪ Digit preference in the recording of vital signs in electronic healthcare records is common, affecting approximately 10% of temperature measurements and 1% of blood pressure recordings in a large UK teaching hospital group ▪ These findings were obtained in hospitals using a semi-automated data capture system that required manual re-entry of vital signs into a tablet computer prior to automated upload to electronic patient records ▪ Digit preference was associated with patient characteristics and was more common in older and male patients, as length of stay increased and following previously normal vital signs ▪ Digit preference varied between hospitals, but decreased over time ▪ Digit preference was generally more common in medical compared to surgical specialties What this adds to what is known ▪ Most previous studies of data quality in electronic patient records have focused on the accuracy of coding ▪ This study focuses on the accuracy of numeric values in patient records, and also adds new data on patient and hospital factors associated with the accuracy of values in electronic patient records Implications ▪ Clinicians and researchers need to be aware that vital signs may not always be accurately documented ▪ Appropriate allowances and adjustments for digit preference should be considered in observational analyses using these factors as outcomes or exposures. ▪ Further work is required to understand the mechanisms behind values preference on a systems, patient and clinician level
BibTeX
@misc{jackson2022quality,
  author = {Niall Jackson and Jessica E. Woods and Peter Watkinson and Andrew Brent and Tim Peto and A. Sarah Walker and David W. Eyre},
  title = {{The quality of vital signs measurements in electronic medical records varies by hospital, specialty, and patient demographics}},
  year = {2022},
  doi = {10.1101/2022.01.19.22269544},
  howpublished = {medRxiv},
}
Figure