Introduction
Imaging photoplethysmography (iPPG) is a non-contact method for measuring vital signs, such as heart rate, by detecting subtle color changes in the skin caused by blood flow. While deep learning has significantly advanced iPPG, these models often suffer from poor generalization due to a "data desert." Unlike standard computer vision tasks with millions of samples, iPPG datasets typically contain only a few hundred videos. This scarcity leads to severe overfitting, especially when models encounter challenging conditions like head motion, varying skin tones, or video compression.
The Challenge of Data Augmentation in iPPG
In traditional computer vision, data augmentation techniques like rotating, flipping, or cropping images are standard practice to improve robustness. However, in the context of iPPG, these "standard" augmentations can be counter-productive. They emphasize large-scale spatial variations that are unrelated to the physiological signal, potentially distracting the model from the faint, temporal color changes it needs to detect. To address this, a more physiologically-aware approach is required.
Methodology: Video Magnification as Augmentation
The authors propose using Video Magnification to artificially amplify the pulse signal within training data. By "zooming in" on the temporal color changes, the model is presented with a clearer version of the target feature, facilitating better learning of the underlying physiology.
The study explores two primary augmentation strategies:
- Heuristic iPPG Augmentation: Using a deep-learning-based magnification tool (DeepMag), the researchers generate multiple versions of each training video with magnification factors ranging from 0.05x to 14x. This exposes the model to a wide variety of signal-to-noise ratios (SNRs).
- Interleaved iPPG Augmentation: A more sophisticated approach where the network is jointly trained to both magnify the video and extract the pulse signal. This method requires a "warm-up" period of standard training (approximately 25 epochs) to ensure the model can accurately identify the pulse before it begins the magnification process.
Experimental Results
The effectiveness of these magnification-based augmentations was evaluated across several public datasets, including AFRL (stationary and motion), MMSE-HR (spontaneous emotion and diverse skin tones), and MR-NIRP (RGB and Near-Infrared).
Key Findings:
- Robustness to Motion: On the AFRL Motion dataset, the Interleaved approach achieved a 46% reduction in Mean Absolute Error (MAE) compared to no augmentation, dropping from 14.39 BPM to 7.73 BPM.
- Performance on Diverse Skin Tones: Darker skin pigmentation (Fitzpatrick scales V-VI) naturally results in lower SNRs. Magnification-based augmentation significantly improved the model's ability to "see" the pulse through higher melanin levels, resulting in a 1.18 dB improvement in SNR.
- Resilience to Compression: Heuristic magnification helped maintain accuracy even under moderate video compression (CRF 24 and 30), where standard models typically fail as the pulse signal is obscured by compression artifacts.
Discussion and Future Work
The study highlights a "Goldilocks" zone for magnification: levels must be high enough to be detectable but low enough to avoid introducing artificial artifacts. While the Interleaved approach is powerful, it is sensitive to the timing of magnification during training; premature magnification can lead to the amplification of noise rather than the pulse.
Future research could focus on automating the selection of optimal magnification levels for specific video conditions and further improving robustness against extreme video compression (e.g., CRF 36), where the physiological signal is almost entirely lost. By treating magnification as a "teacher," this work provides a path forward for training robust physiological sensing models in data-constrained environments.
Summary updated