High-resolution motion-compensated imaging photoplethysmography for remote heart rate monitoring

Audrey G. Chung, Xiaoyu Wang, Robert Amelard, Christian Scharfenberger, Joanne Leong, Jan Kulinski, Alexander Wong, David A. ClausiProceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE 2015 · DOI

TL;DR

This paper introduces a novel non-contact method for remote heart rate monitoring using high-resolution video. By compensating for body motion and analyzing subtle skin color changes (erythema fluctuations), their system achieves improved accuracy compared to existing video-based techniques, paving the way for more accessible health monitoring.

What Problem Does This Paper Address?

Traditional photoplethysmography (PPG) for heart rate monitoring requires physical contact, limiting its use in certain scenarios. While non-contact PPG imaging has emerged, it often struggles with motion artifacts and ambient noise, making reliable heart rate extraction challenging, especially in real-world conditions. Existing methods like Eulerian Video Magnification (EVM) can amplify all changes, including noise, while Independent Component Analysis (ICA) can yield inconsistent results. This paper aims to bridge this gap by developing a more robust and biologically-inspired non-contact PPG system that can accurately measure heart rate even with subject movement.

Methods

The core of the proposed system is a motion-compensated erythema fluctuation analysis. Here's a breakdown of the key steps:

  1. Input Video: High-resolution video of a subject in natural ambient light is captured.
  2. Facial Recognition: A Viola-Jones algorithm is used to detect the subject's face and key facial features (eyes, mouth, nose).
  3. Motion Compensation: A single sample point on the subject's upper cheek is automatically identified and tracked over time using the Kanade-Lucas-Tomasi (KLT) algorithm. This point tracking compensates for natural human motion and lighting fluctuations.
  4. Erythema Fluctuation Analysis: The system analyzes the red and green color channels within a small window around the tracked point. Skin erythema (redness) is known to correlate with blood flow. By analyzing the fluctuations in erythema over time, the system extracts a PPG signal. The specific calculation involves a logarithmic transformation of the ratio of red to green channel intensities.
  5. Heart Rate Estimation: The frequency domain representation of the erythema fluctuation signal is analyzed. The frequency corresponding to the highest amplitude within a plausible heart rate range (40-100 bpm) is identified as the estimated heart rate.

Datasets

The paper describes an experimental setup using eight test videos, each 11-16 seconds long, recorded from five human subjects at rest. These videos were captured using a static mobile phone (HTC One S) in natural ambient light. The ground truth for heart rate measurements was obtained using a consumer-level pulse oximeter (Easy Pulse Sensor version 1.1). No new benchmark datasets were introduced; existing videos were used for evaluation.

Key Insights and Ideas

  • Biologically Inspired Approach: Leveraging skin erythema fluctuations as a proxy for blood flow is a key biological insight that provides a more intuitive and potentially robust signal compared to purely signal-processing-based methods.
  • Motion Compensation is Crucial: The paper strongly emphasizes that compensating for even natural human motion is essential for reliable non-contact PPG. Their KLT-based point tracking effectively addresses this.
  • Targeted Analysis: Focusing the analysis on a small, tracked window on the subject's face, rather than the entire face or frame, significantly reduces the impact of background noise and non-facial motion.
  • Erythema as a Reliable Indicator: The results suggest that skin erythema fluctuations, when properly captured and analyzed, provide a strong signal for heart rate estimation, even in the presence of ambient light and minor motion.

Key Results

The proposed Erythema Fluctuation Analysis (EFA) method was compared against Eulerian Video Magnification (EVM) and Independent Component Analysis (ICA) using the collected test videos.

  • Lowest Percentage Error: The EFA method achieved the lowest mean percentage error (12.1% ± 10.6 std) compared to EVM (28.9% ± 16.6 std) and ICA (20.0% ± 17.3 std).
  • Lowest Standard Deviation: EFA also demonstrated the lowest standard deviation, indicating more consistent and reproducible results.
  • Statistical Significance: A t-test showed a statistically significant difference (p-value of 3.3%) in percentage error distributions between the proposed EFA method and EVM, confirming EFA's superior performance.

Limitations and Future Work

Limitations: * The study was conducted with a small number of subjects (five) and limited video data (eight videos). * All test videos were recorded with subjects at rest, meaning the system's performance under significant physical activity is not yet evaluated. * The experiments were performed under natural ambient light, and the robustness to extreme lighting conditions is not fully explored.

Future Work: * Conduct more extensive testing with a larger and more diverse subject pool. * Evaluate the system's performance under varying lighting conditions. * Incorporate more robust and dynamic sampling methods. * Explore the feasibility of using EFA to capture other cardiovascular metrics, such as heart rate variability and breathing rate.

Summary updated

Abstract

We present a novel non-contact photoplethysmographic (PPG) imaging system based on high-resolution video recordings of ambient reflectance of human bodies that compensates for body motion and takes advantage of skin erythema fluctuations to improve measurement reliability for the purpose of remote heart rate monitoring. A single measurement location for recording the ambient reflectance is automatically identified on an individual, and the motion for the location is determined over time via measurement location tracking. Based on the determined motion information motion-compensated reflectance measurements at different wavelengths for the measurement location can be acquired, thus providing more reliable measurements for the same location on the human over time. The reflectance measurement is used to determine skin erythema fluctuations over time, resulting in the capture of a PPG signal with a high signal-to-noise ratio. To test the efficacy of the proposed system, a set of experiments involving human motion in a front-facing position were performed under natural ambient light. The experimental results demonstrated that skin erythema fluctuations can achieve noticeably improved average accuracy in heart rate measurement when compared to previously proposed non-contact PPG imaging systems.
BibTeX
@inproceedings{chung2015high,
  author = {Audrey G. Chung and Xiaoyu Wang and Robert Amelard and Christian Scharfenberger and Joanne Leong and Jan Kulinski and Alexander Wong and David A. Clausi},
  title = {{High-resolution motion-compensated imaging photoplethysmography for remote heart rate monitoring}},
  booktitle = {Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE},
  year = {2015},
  doi = {10.1117/12.2078397},
}
Figure