Wearable sensors in patient acuity assessment in critical care

Jessica Sena, Mohammad Tahsin Mostafiz, Jiaqing Zhang, Andrea Davidson, Sabyasachi Bandyopadhyay, Subhash Nerella, Yuanfang Ren, Tezcan Ozrazgat‐Baslanti, Benjamin Shickel, Tyler J. Loftus, William Robson Schwartz, Azra Bihorac, Parisa RashidiFrontiers in Neurology 2024 · DOI

TL;DR

This paper introduces a novel approach to assessing patient acuity in the ICU by integrating wearable accelerometer data with electronic health records (EHR). Their deep learning models significantly outperform traditional scoring systems like SOFA, demonstrating the value of mobility data for more granular and accurate patient status evaluation.

What Problem Does This Paper Address?

Assessing patient acuity in Intensive Care Units (ICUs) is crucial for timely interventions and efficient resource allocation. However, current methods heavily rely on manual, subjective patient assessments, which are time-consuming, prone to bias, and often miss subtle changes in a patient's condition. Critically, these traditional methods don't capture dynamic aspects like patient mobility, which can be a key indicator of recovery or deterioration. This gap leaves room for errors and missed opportunities for early intervention, especially in the high-stakes, fast-paced ICU environment.

Methods

The researchers developed an end-to-end deep learning pipeline to assess patient acuity. They utilized data from wrist-worn accelerometers to capture patient mobility and combined this with clinical data from EHRs, including demographics and vital signs. Five deep learning architectures were evaluated: VGG, ResNet, MobileNet, SqueezeNet, and a custom Transformer network. These models were trained to classify patients as either "stable" or "unstable," with instability defined by the need for life-supporting therapies. The performance of these deep learning models was benchmarked against the Sequential Organ Failure Assessment (SOFA) score, a widely used rule-based clinical scoring system.

Datasets

The study used data collected from 87 adult patients admitted to specialized ICUs at the University of Florida Health Shands Hospital. Accelerometer data was collected for up to 7 days per patient, resulting in approximately 9,286 hours of data. Clinical data was extracted from the patients' EHRs. The dataset was split into a development set (70%) for training and hyperparameter tuning, and a holdout test set (30%) for final evaluation, ensuring subject independence.

Key Insights and Ideas

  • Mobility is a critical, underutilized indicator: The study strongly suggests that patient mobility, captured by accelerometers, provides valuable insights into acuity that traditional methods miss.
  • Deep learning excels beyond rule-based systems: Deep learning models, especially when integrating diverse data sources, significantly outperform established clinical scores like SOFA.
  • Multimodal data fusion is key: Combining accelerometer data with demographics and clinical information yields the best predictive performance, highlighting the synergistic benefits of a holistic approach.
  • Granular assessment is possible: The research moves beyond a simple stable/unstable binary, aiming to capture nuances in a patient's physiological and functional status.

Key Results

The deep learning models consistently outperformed the SOFA score baseline. The most comprehensive model, integrating accelerometer data, demographics, and clinical information (Accel + Demo + Clinical), achieved the highest Area Under the Receiver Operating Characteristic (ROC) Curve (AUC) of 0.73. This is a significant improvement compared to the SOFA score's AUC of 0.53. Precision also saw a substantial increase from 0.23 (SOFA) to 0.80 (best model), along with improvements in sensitivity (0.30 vs. 0.60) and F1 score (0.66 vs. 0.77).

Limitations and Future Work

The authors acknowledge several limitations:

  • Generalizability: The study was conducted at a single center with a predominantly white patient population, necessitating validation on more diverse datasets.
  • Data collection nuances: While efforts were made to ensure continuous data, some minor data loss due to device removal is possible.
  • Intubated/sedated patients: The utility of mobility data for assessing acuity in intubated and sedated patients, who have extremely limited active mobility, requires further investigation.
  • SHAP interpretation: SHAP feature importance is correlated with model performance and can be influenced by overfitting.

Future work suggested includes:

  • Integrating additional clinical features (e.g., medication history, lab results).
  • Exploring multimodal sensing beyond accelerometers, such as depth images, RGB images, and electromyography.
  • Investigating seamless integration into clinical workflows to avoid adding to nursing workload.
  • Developing effective ways to communicate model outputs and assessments to healthcare providers.

Summary updated

Abstract

Acuity assessments are vital for timely interventions and fair resource allocation in critical care settings. Conventional acuity scoring systems heavily depend on subjective patient assessments, leaving room for implicit bias and errors. These assessments are often manual, time-consuming, intermittent, and challenging to interpret accurately, especially for healthcare providers. This risk of bias and error is likely most pronounced in time-constrained and high-stakes environments, such as critical care settings. Furthermore, such scores do not incorporate other information, such as patients' mobility level, which can indicate recovery or deterioration in the intensive care unit (ICU), especially at a granular level. We hypothesized that wearable sensor data could assist in assessing patient acuity granularly, especially in conjunction with clinical data from electronic health records (EHR). In this prospective study, we evaluated the impact of integrating mobility data collected from wrist-worn accelerometers with clinical data obtained from EHR for estimating acuity. Accelerometry data were collected from 87 patients wearing accelerometers on their wrists in an academic hospital setting. The data was evaluated using five deep neural network models: VGG, ResNet, MobileNet, SqueezeNet, and a custom Transformer network. These models outperformed a rule-based clinical score (Sequential Organ Failure Assessment, SOFA) used as a baseline when predicting acuity state (for ground truth we labeled as unstable patients if they needed life-supporting therapies, and as stable otherwise), particularly regarding the precision, sensitivity, and F1 score. The results demonstrate that integrating accelerometer data with demographics and clinical variables improves predictive performance compared to traditional scoring systems in healthcare. Deep learning models consistently outperformed the SOFA score baseline across various scenarios, showing notable enhancements in metrics such as the area under the receiver operating characteristic (ROC) Curve (AUC), precision, sensitivity, specificity, and F1 score. The most comprehensive scenario, leveraging accelerometer, demographics, and clinical data, achieved the highest AUC of 0.73, compared to 0.53 when using SOFA score as the baseline, with significant improvements in precision (0.80 vs. 0.23), specificity (0.79 vs. 0.73), and F1 score (0.77 vs. 0.66). This study demonstrates a novel approach beyond the simplistic differentiation between stable and unstable conditions. By incorporating mobility and comprehensive patient information, we distinguish between these states in critically ill patients and capture essential nuances in physiology and functional status. Unlike rudimentary definitions, such as equating low blood pressure with instability, our methodology delves deeper, offering a more holistic understanding and potentially valuable insights for acuity assessment.
BibTeX
@inproceedings{sena2024wearable,
  author = {Jessica Sena and Mohammad Tahsin Mostafiz and Jiaqing Zhang and Andrea Davidson and Sabyasachi Bandyopadhyay and Subhash Nerella and Yuanfang Ren and Tezcan Ozrazgat‐Baslanti and Benjamin Shickel and Tyler J. Loftus and William Robson Schwartz and Azra Bihorac and Parisa Rashidi},
  title = {{Wearable sensors in patient acuity assessment in critical care}},
  booktitle = {Frontiers in Neurology},
  year = {2024},
  doi = {10.3389/fneur.2024.1386728},
}
Figure