TRUST: Time-Domain Residual Unsupervised Stability Technique for Improved Heart Rate Estimation

Shahzad Ahmad, Sania Bano, Sukalpa Chanda, Santosh Kumar Vipparthi, Subrahmanyam MuralaWACV 2025

TL;DR

This paper introduces TRUST, a novel unsupervised learning framework for estimating heart rate from facial videos. By incorporating time-domain loss functions and feature stabilization blocks, TRUST significantly improves the robustness and accuracy of rPPG signals, especially in challenging low-level feature extraction scenarios, outperforming existing state-of-the-art unsupervised methods.

What Problem Does This Paper Address?

Estimating physiological signals like heart rate (HR) from video (remote photoplethysmography or rPPG) holds immense promise for non-contact health monitoring. However, a major bottleneck is the scarcity of synchronized video and ground-truth physiological data, making it difficult to train supervised deep learning models. While unsupervised learning offers a solution, existing methods often struggle with unstable low-level spatial features, leading to noisy rPPG signals. This paper aims to address this by enhancing unsupervised rPPG estimation through improved feature stability.

Methods

The core of TRUST builds upon the non-contrastive unsupervised learning framework (SiNC) introduced by Speth et al. [32]. The key innovations are:

  • Time-Domain Loss Functions: The authors introduce novel temporal loss functions (Temporal Variance Loss, Signal Energy Loss, and Derivative Consistency Loss) that operate in the time domain. These losses aim to regularize the signal, suppress abrupt variations, and encourage smoother, more consistent transitions, complementing existing frequency-domain losses.
  • Feature Stabilization Blocks: To directly address the issue of unstable low-level spatial features, TRUST incorporates dedicated "Feature Stabilization Blocks" (Sᵢ). These blocks are strategically placed parallel to the feature extraction blocks (Bᵢ) within the network architecture. They use 1x1 3D convolutions and batch normalization to stabilize spatial features at various downsampling stages, while residual connections ensure high-level information is retained.
  • Feature Stabilization Metric (ρ): A new metric is proposed to quantitatively assess feature instability in the model's last layer at a low spatial resolution (4x4). This metric correlates with MAE, providing insight into the model's interpretability and the reliability of its extracted features.

The overall loss function is a combination of frequency-domain losses (Lfreq) and the newly introduced time-domain losses (Ltime).

Datasets

The paper uses four publicly available datasets for training and evaluation:

  • PURE [34]: 60 face videos from 10 subjects, recorded over six sessions with diverse head motions.
  • UBFC-rPPG [1]: 42 one-minute face videos from subjects performing a math game.
  • COHFACE [12]: 160 one-minute videos from 40 subjects, captured under various lighting conditions and with significant video compression.
  • BH-rPPG [43]: 105 videos from 35 subjects, captured under three different lighting conditions.

The authors employed a 5-fold cross-validation approach, using three folds for training, one for validation, and one for testing, consistent with previous work [11].

Key Insights and Ideas

  • Time-Domain Stability is Crucial: The paper strongly emphasizes that while frequency-domain analysis is important for rPPG, temporal stability in low-level features is a critical, often overlooked, factor for accurate signal extraction.
  • Unsupervised Learning Can Compete with Supervised Methods: By focusing on feature stabilization and incorporating appropriate loss functions, TRUST demonstrates that unsupervised methods can achieve performance comparable to, and sometimes exceeding, supervised approaches, especially in challenging scenarios.
  • Feature Stabilization Blocks are Effective: The introduction of dedicated blocks to stabilize features at lower spatial resolutions proved to be a key enabler for TRUST's improved performance. This suggests that targeted architectural modifications can significantly boost robustness.
  • A New Metric for Interpretability: The feature stabilization metric (ρ) offers a valuable tool for understanding why a model performs well or poorly, by directly quantifying the stability of learned features.

Key Results

  • Intra-Dataset Performance: TRUST consistently outperformed other unsupervised methods, including the SiNC baseline, on the PURE and UBFC datasets, achieving low MAE and high Pearson correlation (r close to 1). On COHFACE, TRUST surpassed all supervised and unsupervised methods. For BH-rPPG, its performance was comparable to the best supervised methods.
  • Cross-Dataset Performance: TRUST showed good robustness when transferring knowledge across datasets, with notable performance when training on PURE and testing on UBFC, and vice-versa. However, performance degraded when testing on datasets with different frame rates (COHFACE and BH-rPPG), highlighting the impact of this factor.
  • Robustness to Conditions: On the COHFACE dataset, TRUST demonstrated superior or competitive performance across various conditions (light, skin tone, motion), outperforming the SiNC model.
  • Ablation Study: The ablation study revealed that while individual loss components contributed, combining all three time-domain losses with the feature extraction block yielded the most significant performance enhancement. The feature extraction block itself was identified as crucial for uncovering the rPPG signal.

Limitations and Future Work

  • Cross-Dataset Frame Rate Discrepancies: The paper acknowledges that variations in frame rates across datasets (e.g., COHFACE at 20 FPS, BH-rPPG at 15 FPS) negatively impact cross-dataset generalization. Future work could explore methods to better handle these temporal resolution differences.
  • Computational Cost: While the paper mentions the model is designed for efficiency, the use of 3D convolutions and processing 120 frames might still present computational challenges for real-time deployment on resource-constrained devices.
  • Further Exploration of Augmentations: The paper mentions various augmentations used to improve robustness. Further research could explore the optimal combination and impact of these augmentations.
  • Generalizability to More Diverse Conditions: While tested on several conditions, future work could investigate TRUST's performance in even more extreme scenarios, such as severe occlusions or highly dynamic environments.

Summary updated

BibTeX
@inproceedings{ahmad2025trust,
  author = {Shahzad Ahmad and Sania Bano and Sukalpa Chanda and Santosh Kumar Vipparthi and Subrahmanyam Murala},
  title = {{TRUST: Time-Domain Residual Unsupervised Stability Technique for Improved Heart Rate Estimation}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2025},
}
Figure