Video object detection for privacy-preserving patient monitoring in intensive care

Raphael Emberger, Jens Michael Boss, Daniel Baumann, Marko Seric, Shufan Huo, Lukas Tuggener, Emanuela Keller, Thilo Stadelmann

Abstract

Patient monitoring in intensive care units (ICUs) is essential for patient safety but places a significant burden on clinical staff. While automated clinical decision support systems aim to alleviate this, they are often prone to false alarms caused by artifacts—such as patient movement—that mimic critical biosignals. Video monitoring offers a solution for artifact classification, yet privacy regulations necessitate heavy blurring, which degrades the performance of standard object detection (OD) models. This paper proposes a novel "channel-repurposing" method that embeds temporal information directly into the input channels of a standard YOLOv5 architecture. By replacing RGB channels with grayscale structure, pixel-wise motion, and historical detection data, our approach improves detection accuracy by 1.7% [email protected] and reduces training time by over 90% compared to baseline models. This method provides a computationally efficient, privacy-preserving solution suitable for deployment on standard hospital IT infrastructure.

Introduction

Intensive Care Units are high-stakes environments where medical staff frequently experience "alarm fatigue" due to a high volume of false-positive alerts. These alerts are often triggered by biosignal artifacts; for instance, a patient adjusting their position may be misinterpreted by sensors as a cardiac arrhythmia.

Integrating video data could provide the necessary context to filter these artifacts, but two primary hurdles remain: privacy and hardware constraints. To protect the dignity of patients and staff, video must be stored with significant blurring, stripping away the fine visual details that standard AI models require. Furthermore, hospital security protocols often require that these systems run locally on modest hardware, making computationally expensive video models impractical. This research explores a way to grant a lightweight, off-the-shelf image detector temporal awareness without increasing its architectural complexity.

Methodology

The core innovation of this study is the repurposing of the three standard image color channels (Red, Green, and Blue) to carry temporal and structural information. Since color is largely lost or irrelevant in heavily blurred ICU footage, the researchers modified the input as follows:

  • Red Channel (Structure): Contains a grayscale version of the current frame to preserve the general shapes and spatial layout of the room.
  • Green Channel (Motion): Contains a pixel-wise difference map between the current and previous frames. This highlights movement, allowing the model to distinguish between stationary furniture and moving personnel.
  • Blue Channel (Memory): Contains a bitmap of bounding boxes from the previous frame. This "nudges" the model to maintain temporal consistency by looking where objects were recently located.

To ensure the model did not become overly reliant on the "Memory" channel, the team employed data augmentation techniques, such as randomly dropping or shifting the previous bounding boxes during training. This forced the model to prioritize visual evidence from the Red and Green channels while using the Blue channel as a secondary guide.

Experimental Setup

The study utilized a proprietary dataset from a 12-bed neurocritical care unit at the University Hospital Zurich. The footage (640x400 resolution at 25 fps) was processed with a heavy 6:1 box blur to ensure privacy. The dataset consisted of 30,748 video clips extracted via motion detection, with 196 clips hand-labeled for four specific classes: patient, bed, staff, and medical devices.

Results and Discussion

The proposed method was benchmarked against a standard YOLOv5 baseline. The results demonstrated both improved accuracy and a dramatic increase in training efficiency:

  • Detection Accuracy: The proposed model achieved 88.9% [email protected], a 1.7% improvement over the 87.2% achieved by the baseline. The model was particularly effective at identifying "beds" and "patients," though "staff" remained more difficult to classify due to erratic movement patterns.
  • Training Efficiency: The repurposed channels provided such clear indicators of object location that the model converged in just 10 epochs, compared to 119 epochs for the baseline. This represents a training time reduction of over 91%.
  • Qualitative Analysis: Confusion matrices indicated that the model became significantly better at distinguishing relevant objects from the background, likely because the motion channel effectively filtered out static environmental noise.

Conclusion

By repurposing standard image channels to encode temporal data, we have demonstrated that lightweight object detectors can be highly effective even in low-information, blurred environments. This approach allows for the deployment of sophisticated monitoring tools within the strict privacy and hardware constraints of a hospital setting. While the current method is optimized for stationary cameras, future work will explore adding additional channels to retain color information and testing the system's robustness against camera movement.

Summary updated

Abstract

Patient monitoring in intensive care units, although assisted by biosensors, needs continuous supervision of staff. To reduce the burden on staff members, IT infrastructures are built to record monitoring data and develop clinical decision support systems. These systems, however, are vulnerable to artifacts (e.g. muscle movement due to ongoing treatment), which are often indistinguishable from real and potentially dangerous signals. Video recordings could facilitate the reliable classification of biosignals using object detection (OD) methods to find sources of unwanted artifacts. Due to privacy restrictions, only blurred videos can be stored, which severely impairs the possibility to detect clinically relevant events such as interventions or changes in patient status with standard OD methods. Hence, new kinds of approaches are necessary that exploit every kind of available information due to the reduced information content of blurred footage and that are at the same time easily implementable within the IT infrastructure of a normal hospital. In this paper, we propose a new method for exploiting information in the temporal succession of video frames. To be efficiently implementable using off-the-shelf object detectors that comply with given hardware constraints, we repurpose the image color channels to account for temporal consistency, leading to an improved detection rate of the object classes. Our method outperforms a standard YOLOv5 baseline model by +1.7% [email protected] while also training over ten times faster on our proprietary dataset. We conclude that this approach has shown effectiveness in the preliminary experiments and holds potential for more general video OD in the future.
BibTeX
@inproceedings{emberger2023video,
  author = {Raphael Emberger and Jens Michael Boss and Daniel Baumann and Marko Seric and Shufan Huo and Lukas Tuggener and Emanuela Keller and Thilo Stadelmann},
  title = {{Video object detection for privacy-preserving patient monitoring in intensive care}},
  booktitle = {SDS},
  year = {2023},
  url = {https://openreview.net/forum?id=KyFyqDZt0h},
}
Figure