Introduction
Remote photoplethysmography (rPPG) has emerged as a powerful non-contact method for measuring physiological signals, such as heart rate, by detecting subtle color changes in skin video. While deep learning architectures, particularly 3D Convolutional Neural Networks (3DCNNs), have significantly advanced the field, their design remains largely a "black box" process. Engineers often struggle to determine the optimal depth for these models, leading to either under-powered architectures that miss critical features or over-engineered ones that waste computational resources. This paper explores the use of Centered Kernel Alignment (CKA) as a diagnostic tool to visualize internal representations and optimize rPPG model depth.
Methodology
The researchers employed Centered Kernel Alignment (CKA) to compare the internal features learned by different layers within a network. By analyzing the similarity of these representations, it is possible to identify redundant layers and functional transitions within the architecture.
Two primary state-of-the-art architectures were evaluated:
- PhysNet-3DCNN: A volume-processing model typically configured with 10 layers.
- TS-CAN (Temporal Shift Convolutional Attention Network): A dual-branch model designed to separate spatial and temporal information, originally using a shallow 2-layer design.
These models were tested across five diverse datasets (PURE, UBFC-rPPG, MSPM, DDPM, and BP4D+) to ensure the generalizability of the findings across varying levels of motion and environmental complexity.
Analyzing Internal Representations
CKA analysis revealed distinct structural behaviors in the two models. In the 3DCNN, the network naturally organizes into functional blocks: early layers focus on spatial skin detection, while later layers transition to temporal pulse extraction. However, the analysis also uncovered significant redundancy in the standard 10-layer PhysNet, where large blocks of layers learned nearly identical features.
In contrast, TS-CAN exhibited a much sharper diagonal in its CKA plots, as seen in Figure 2(b). This indicates that each layer in the TS-CAN architecture learns a unique representation, suggesting that the original shallow design was under-provisioned and could benefit from additional depth.
Empirical Validation and Results
The theoretical insights provided by CKA were validated through extensive empirical testing. By measuring the Mean Absolute Error (MAE) across varying model depths, the researchers confirmed that CKA-identified redundancies correlate directly with performance plateaus.
As shown in Figure 6, the 3DCNN's performance plateaus after approximately 5 to 6 layers. A 5-layer PhysNet achieves comparable accuracy to the 10-layer version while offering significant improvements in training speed and inference latency. Conversely, for TS-CAN, increasing depth from 2 to 5 meta-layers consistently reduced error rates across all datasets, confirming that the model was previously under-utilized.
Conclusion
This study demonstrates that CKA is an effective diagnostic tool for "X-raying" rPPG neural networks. By identifying redundant layers in PhysNet and untapped potential in TS-CAN, the researchers were able to refine these architectures for better efficiency and accuracy. Future work will focus on stabilizing deeper TS-CAN models using techniques like skip connections and exploring CKA's utility in optimizing other hyperparameters, such as kernel size and channel count.
Summary updated