Contactless Pulse Estimation Leveraging Pseudo Labels and Self-Supervision

Zhihua Li, Lijun YinICCV 2023 · DOI

Introduction

Remote photoplethysmography (rPPG) is a non-contact method for measuring heart rate by detecting subtle skin color changes in facial videos. While deep learning has significantly advanced this field, most high-performing models rely on large datasets with synchronized medical-grade ground truth (ECG/PPG). Acquiring such labels is expensive and difficult. This paper proposes a framework that achieves state-of-the-art performance without any manual labels by leveraging self-supervision and a novel label correction strategy.

Methodology: Bridging the Gap

The proposed approach uses a three-pronged strategy to move from noisy, unsupervised estimates to accurate pulse predictions.

1. Spectral Ranking Loss

Traditional self-supervised learning often fails in rPPG because models get distracted by non-pulse periodic signals like head movement or video compression. The authors introduce a Spectral Ranking Loss. By artificially speeding up or slowing down video segments, they force the network to predict heart rates that scale accordingly. This ensures the model learns the intrinsic frequency of the pulse rather than static environmental noise.

2. Guided Pseudo-Labeling

The process begins with "pseudo-labels" generated by a traditional, non-learning-based algorithm called 2SR. While 2SR is robust, it is often imprecise. The training follows a "fit-and-escape" strategy: the model first fits to these rough pseudo-labels to find a reasonable starting point in the search space, then "escapes" their inaccuracies through self-supervised fine-tuning.

3. Co-Rectification (Collaborative Label Correction)

To handle the inherent noise in pseudo-labels, the framework employs Co-Rectification. Two separate networks are trained simultaneously. When one network's prediction deviates significantly from the pseudo-label, the system assumes the pseudo-label is erroneous and replaces it with the prediction from the peer network.

The Co-Rectification framework uses two models to identify and replace noisy pseudo-labels during training.

Experimental Results

The method was evaluated on several benchmarks, including the challenging VIPL-HR and V4V datasets.

Performance and Robustness

On the VIPL-HR dataset, the proposed method achieved an RMSE of 8.26, significantly outperforming the previous unsupervised state-of-the-art (Gideon2021, RMSE 14.02). Remarkably, on the V4V dataset, it matched or exceeded the performance of fully supervised models like PhysFormer.

Comparison of pseudo-labels (left) and network predictions (right) against ground truth. The network effectively filters out the noise present in the initial labels.

A key strength of this approach is its robustness to video compression. While traditional unsupervised methods often fail when video quality drops, this framework maintains high accuracy even under "Ultra high" compression ratios.

Conclusion

By combining physics-based spectral constraints with a collaborative learning strategy, this work demonstrates that manual labeling is no longer a bottleneck for high-accuracy rPPG. Future work may extend this "label-free" paradigm to other physiological metrics like respiration and blood oxygen levels.

Summary updated

BibTeX
@inproceedings{li2023contactless,
  author = {Zhihua Li and Lijun Yin},
  title = {{Contactless Pulse Estimation Leveraging Pseudo Labels and Self-Supervision}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2023},
  doi = {10.1109/iccv51070.2023.01882},
}
Figure