Video Object Detection for Privacy-Preserving Patient Monitoring in Intensive Care

Raphael Emberger, Jens Michael Boss, Daniel Baumann, Marko Seric, Shufan Huo, Lukas Tuggener, Emanuela Keller, Thilo Stadelmann

TL;DR

Researchers have developed a clever "channel-repurposing" technique that allows standard, lightweight object detectors to "see" temporal movement in blurred, privacy-preserving ICU footage. By replacing traditional RGB channels with grayscale, motion, and historical detection data, the team improved detection accuracy by 1.7% mAP while slashing training time by over 90% compared to a standard YOLOv5 baseline.

What Problem Does This Paper Address?

Intensive Care Units (ICUs) are high-stakes environments where medical staff are frequently overwhelmed by "alarm fatigue." Many of these alarms are false positives triggered by biosignal artifacts—for example, a patient moving their arm might look like a heart rhythm abnormality to a sensor.

While video monitoring could provide the context needed to filter out these artifacts, it faces two major hurdles:

  1. Privacy: To protect patient and staff dignity, video must be stored and processed with heavy blurring, which strips away the fine visual details standard AI models rely on.
  2. Hardware Constraints: For security and privacy, these systems must run locally on hospital hardware, ruling out massive, computationally expensive "heavyweight" video models.

The authors sought a way to give a lightweight, "off-the-shelf" image detector (YOLOv5) temporal awareness—the ability to understand movement over time—without changing the model's architecture or increasing its computational footprint.

Methods

The core innovation is a "hack" of the input data. Standard images have three color channels: Red, Green, and Blue (RGB). Since color is largely irrelevant in heavily blurred ICU footage, the researchers repurposed these channels to carry temporal information:

  • Red Channel (Structure): Replaced with a grayscale version of the current frame. This preserves the general shapes and structures of the room.
  • Green Channel (Motion): Replaced with a pixel-wise difference map between the current and previous frames. This highlights movement, helping the model distinguish between a stationary bed and a moving staff member.
  • Blue Channel (Memory): Replaced with a bitmap of bounding boxes from the previous frame (either ground truth during training or the model's own previous predictions). This "nudges" the model to look where objects were just a moment ago, enforcing temporal consistency.

To prevent the model from becoming "lazy" and over-relying on the Blue channel, they used data augmentation—randomly dropping or shifting the previous bounding boxes—forcing the model to actually learn from the visual data.

Datasets

The study used a proprietary dataset collected from a 12-bed neurocritical care unit at the University Hospital Zurich.

  • Data Specs: 640x400 resolution at 25 fps, processed with a heavy "box blur" (6:1 ratio) to ensure privacy.
  • Scale: 30,748 video clips were automatically extracted based on motion detection.
  • Ground Truth: 196 clips were hand-labeled with oriented bounding boxes for four classes: patient, bed, staff, and medical devices.

Key Insights and Ideas

  • Motion > Color: In low-information environments (like blurred video), the change between frames is often more informative than the static pixels of a single frame.
  • Architecture Stability: By keeping the YOLOv5 architecture identical and only changing the input data, the researchers ensured the model remained compatible with existing hospital IT infrastructure and specialized hardware accelerators.
  • Training Efficiency: The most surprising finding was the massive jump in training speed. Because the repurposed channels provided such clear "hints" about where to look (via the motion and memory channels), the model converged in just 10 epochs, compared to 119 for the baseline.
  • Contextual Artifact Removal: The authors argue that detecting a "staff member" near a "medical device" provides the necessary context to explain why a biosignal might suddenly spike, allowing for smarter alarm systems.

Key Results

The proposed method was compared against a standard YOLOv5 baseline:

  • Accuracy: The proposed model achieved 88.9% [email protected], a +1.7% improvement over the baseline (87.2%).
  • Category Performance: The model showed significant gains in detecting "beds" and "patients," though "staff" remained a challenge due to their high variability in movement and appearance.
  • Training Speed: The proposed method required only 8.4% of the training time used by the baseline (10 epochs vs. 119 epochs).
  • Qualitative: The confusion matrices show the model is much better at distinguishing the background from relevant objects, likely due to the motion channel highlighting what is actually moving.

Limitations and Future Work

The current method relies on a stationary camera. Because the "Green channel" (motion) is a simple pixel difference, it would likely fail if the camera itself were moving (panning or tilting), as the entire frame would appear to be in motion.

Future work suggested by the authors includes:

  • Expanding Channels: Instead of replacing RGB, they suggest adding more channels to the input to see if retaining some color information helps.
  • Non-Stationary Testing: Evaluating how the system handles changes in perspective or viewing angles.
  • Ablation Studies: Conducting deeper dives into which of the three repurposed channels contributes the most to the performance gain.

Summary updated

Abstract

Patient monitoring in intensive care units, although assisted by biosensors, needs continuous supervision of staff. To reduce the burden on staff members, IT infrastructures are built to record monitoring data and develop clinical decision support systems. These systems, however, are vulnerable to artifacts (e.g. muscle movement due to ongoing treatment), which are often indistinguishable from real and potentially dangerous signals. Video recordings could facilitate the reliable classification of biosignals using object detection (OD) methods to find sources of unwanted artifacts. Due to privacy restrictions, only blurred videos can be stored, which severely impairs the possibility to detect clinically relevant events such as interventions or changes in patient status with standard OD methods. Hence, new kinds of approaches are necessary that exploit every kind of available information due to the reduced information content of blurred footage and that are at the same time easily implementable within the IT infrastructure of a normal hospital. In this paper, we propose a new method for exploiting information in the temporal succession of video frames. To be efficiently implementable using off-the-shelf object detectors that comply with given hardware constraints, we repurpose the image color channels to account for temporal consistency, leading to an improved detection rate of the object classes. Our method outperforms a standard YOLOv5 baseline model by +1.7% [email protected] while also training over ten times faster on our proprietary dataset. We conclude that this approach has shown effectiveness in the preliminary experiments and holds potential for more general video OD in the future.
BibTeX
@inproceedings{emberger2023video,
  author = {Raphael Emberger and Jens Michael Boss and Daniel Baumann and Marko Seric and Shufan Huo and Lukas Tuggener and Emanuela Keller and Thilo Stadelmann},
  title = {{Video Object Detection for Privacy-Preserving Patient Monitoring in Intensive Care}},
  booktitle = {SDS},
  year = {2023},
  url = {https://openreview.net/forum?id=KyFyqDZt0h},
}

Loading related content...

Figure