Chaos in Motion: Unveiling Robustness in Remote Heart Rate Measurement through Brain-Inspired Skin Tracking

Jie Wang, Jing Lian, Minjie Ma, Junqiang Lei, Chunbiao Li, Bin Li, Jizhao LiuarXiv 2024 · DOI · arXiv

Abstract

Remote photoplethysmography (rPPG) enables non-contact heart rate monitoring by detecting subtle skin color changes in video. However, traditional methods rely heavily on face detection, making them vulnerable to motion, specific clinical conditions, and privacy concerns. This paper introduces HR-RST, a framework utilizing a brain-inspired Continuous Coupled Neural Network (CCNN) to track skin pixels based on their chaotic signal characteristics. By decoupling heart rate measurement from facial landmarks, the system achieves robustness against movement and extends monitoring to non-facial body parts, enhancing both clinical utility and biometric privacy.

Introduction

Remote heart rate measurement (rPPG) is a technique that extracts a pulse signal by analyzing invisible-to-the-eye color changes in skin video. While promising for non-invasive health monitoring, this technology often fails under real-world conditions such as subject movement, lighting variations, or when a face cannot be clearly identified. Traditional rPPG pipelines are inherently fragile because they depend on rigid face-tracking algorithms to define a Region of Interest (ROI).

Overview of the rPPG pipeline and the three primary limitations of traditional face-centric approaches: motion sensitivity, clinical constraints, and privacy risks.

The Limitations of Face-Centric rPPG

Most state-of-the-art rPPG methods follow a rigid sequence: detect the face, define an ROI, and extract the signal. This approach suffers from three major "fail states":

  1. Motion Sensitivity: Rapid head movements or tilts often cause face trackers to lose their grip, resulting in failed or inaccurate signal extraction.
  2. Clinical Constraints: Models trained on healthy adult faces frequently fail for infants or burn victims, whose skin textures or movement patterns differ significantly.
  3. Privacy Risks: The requirement for high-resolution facial video creates substantial biometric privacy concerns in medical monitoring environments.
Detailed view of face detection failure cases where head rotation leads to inaccurate landmark placement or complete detection failure.

The HR-RST Framework

The proposed Heart Rate - Robust Skin Tracking (HR-RST) framework shifts the focus from finding faces to identifying skin based on the unique "shimmer" of the pulse. This brain-inspired approach operates in three distinct phases:

1. ROI Extraction via CCNN

Instead of standard convolutional networks, HR-RST uses a Continuous Coupled Neural Network (CCNN) that mimics the primary visual cortex. The system distinguishes skin from background by analyzing signal characteristics. The CCNN encodes dynamic pixels (skin with a pulse) into chaotic signals, while static background pixels are encoded as periodic signals. By applying a Continuous Wavelet Transform (Gaussian wavelets) to the neuron outputs, the system isolates skin pixels without requiring facial landmark detection.

2. Signal Analysis

Once the skin ROI is isolated, the system processes the Green color channel, which carries the strongest pulse signal due to oxyhemoglobin absorption. A third-order Butterworth band-pass filter (0.7Hz to 2.5Hz) is applied to target the human heart rate range (42–150 bpm).

3. Pixel-wise HR Calculation

Unlike traditional methods that average the entire ROI into a single signal, HR-RST calculates the heart rate for every individual pixel using Power Spectral Density (PSD) estimation via the Welch method. The final heart rate is determined by the mode of these pixel-wise measurements. This approach provides extreme resilience against local noise, shadows, or partial occlusions.

Experimental Evaluation

The framework was validated using the VIPL-HR Database (2,378 videos) and a Custom Non-Facial Dataset featuring body parts such as the palm, forearm, and sole of the foot.

Key Findings

  • Versatility: The system successfully extracted heart rate signals from non-facial areas, enabling "de-identified" monitoring where a patient's face is never recorded.
  • Robustness: The CCNN-based tracker maintained skin contact during significant head rotations (pitch/yaw) where traditional face-detectors failed.
  • Quantitative Performance: On the VIPL-HR dataset, the system showed consistent performance across subjects; for example, Sample p1 achieved a Mean Absolute Error (MAE) of 12.41 bpm. While deep learning models may achieve lower error in controlled settings, HR-RST offers higher interpretability and functions without massive training datasets.

Conclusion

HR-RST demonstrates that treating the pulse as a chaotic perturbation allows for robust, privacy-preserving heart rate monitoring. By moving away from face-centric detection, the system opens new possibilities for clinical monitoring in scenarios where traditional sensors or face-tracking are impractical, such as neonatal care and burn units.

Summary updated

Abstract

Heart rate is an important physiological indicator of human health status. Existing remote heart rate measurement methods typically involve facial detection followed by signal extraction from the region of interest (ROI). These SOTA methods have three serious problems: (a) inaccuracies even failures in detection caused by environmental influences or subject movement; (b) failures for special patients such as infants and burn victims; (c) privacy leakage issues resulting from collecting face video. To address these issues, we regard the remote heart rate measurement as the process of analyzing the spatiotemporal characteristics of the optical flow signal in the video. We apply chaos theory to computer vision tasks for the first time, thus designing a brain-inspired framework. Firstly, using an artificial primary visual cortex model to extract the skin in the videos, and then calculate heart rate by time-frequency analysis on all pixels. Our method achieves Robust Skin Tracking for Heart Rate measurement, called HR-RST. The experimental results show that HR-RST overcomes the difficulty of environmental influences and effectively tracks the subject movement. Moreover, the method could extend to other body parts. Consequently, the method can be applied to special patients and effectively protect individual privacy, offering an innovative solution.
BibTeX
@misc{wang2024chaos,
  author = {Jie Wang and Jing Lian and Minjie Ma and Junqiang Lei and Chunbiao Li and Bin Li and Jizhao Liu},
  title = {{Chaos in Motion: Unveiling Robustness in Remote Heart Rate Measurement through Brain-Inspired Skin Tracking}},
  year = {2024},
  eprint = {2404.07687},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2404.07687},
}
Figure