Introduction
Remote photoplethysmography (rPPG) is a non-contact method for measuring heart rate by detecting subtle skin color changes in facial videos. Despite significant advancements driven by deep learning, the field faces a persistent "ground truth" crisis. This crisis stems from the inherent temporal misalignments between video data and traditional medical sensors, which complicates the training of end-to-end networks.
The Ground Truth Crisis
Standard rPPG datasets typically utilize Electrocardiogram (ECG) or finger-clip Blood Volume Pulse (BVP) sensors as labels. However, an uncertain temporal delay exists between the heart's electrical activity and the blood flow changes visible on the face. Furthermore, the "envelope shape" of these label waves varies significantly between sensors and individuals.
Traditionally, researchers have relied on manual alignment—a process that is time-consuming, error-prone, and often results in discarding a large portion of available data that cannot be perfectly synchronized. This bottleneck limits the utility of existing datasets and the scalability of rPPG models.
The BYHE Framework: Rhythm over Alignment
The proposed WPPG Net, utilizing the BYHE (Boost Your Heartbeat Estimation) framework, addresses this bottleneck by shifting the focus from absolute signal alignment to intrinsic rhythm self-similarity. Instead of attempting to predict a perfectly aligned 1D wave, the network focuses on the periodicity of the signal.
Methodology
The BYHE framework introduces a novel approach to label representation and loss calculation:
- Phase Map Representation: The ground truth is preprocessed using a Butterworth filter and Continuous Wavelet Transform (CWT). A Hilbert transform extracts the instantaneous phase to generate a Target Label Matrix ($R$). Each element $R_{i,j}$ represents the cosine similarity of the phase between two time points, making the representation invariant to constant temporal shifts.
- Network Architecture: The framework is designed as a "graft" that can be added to standard backbone networks (such as the Contrastive Attention Network). It adds an adjustment layer that uses sliding windows to extract feature vectors and calculates their cosine similarity to produce a predicted matrix ($\hat{R}$) that mirrors the structure of the label matrix.
Loss Functions
Training is guided by a composite loss function designed to ensure rhythmic stability:
- $Loss_{MSE}$: Ensures the predicted matrix matches the structure of the label matrix.
- $Loss_{Pearson}$: Maintains rhythm consistency across rows of the matrix.
- $Reg$ (Regularization): A novel term that encourages "diagonal uniformity," ensuring the detected rhythm remains stable over time and preventing the network from collapsing into non-rhythmic predictions.
Experimental Results
The BYHE framework was evaluated on two gold-standard datasets: VIPL-HR and MAHNOB-HCI. By ignoring the alignment struggle, the method utilizes nearly 100% of the dataset, compared to the ~50% often used by traditional methods that require clean, synchronized slices.
- VIPL-HR: Achieved a Mean Absolute Error (MAE) of 4.85 bpm, outperforming specialized models like PhysFormer (4.97 bpm) and Dual-GAN (4.93 bpm).
- MAHNOB-HCI: Achieved an MAE of 3.54 bpm, demonstrating competitive performance against state-of-the-art meta-learning and transformer-based approaches.
Ablation studies confirmed that the regularization term ($reg$) is vital; without it, the network fails to maintain a stable rhythmic signal.
Limitations and Future Work
While BYHE effectively solves the alignment problem, it remains sensitive to extreme noise. In scenarios with fierce motion or lighting changes that completely mask the rPPG signal, the self-similarity matrix becomes unreliable. Future research will focus on hybrid architectures and more precise facial skin masking to further isolate the rPPG signal from environmental noise.
Conclusion
The WPPG Net (BYHE) demonstrates that for heart rate estimation, the interval between beats is more critical than the exact timestamp. By utilizing self-similarity matrices, the network learns the "tempo" of the heart without the confusion caused by sensor delays. This approach significantly improves data efficiency and allows classical architectures to achieve state-of-the-art performance in remote physiological monitoring.
Summary updated