TL;DR
This paper investigates the impact of face blurring on 3D human pose estimation and subsequent kinematic analysis. The authors demonstrate that blurring faces significantly enhances subject privacy without substantially degrading the accuracy of pose estimation or kinematic calculations, making it a viable technique for sensitive applications.
What Problem Does This Paper Address?
Human pose estimation is a critical tool for applications in fields like ergonomics and occupational health, enabling the analysis of human movement to prevent injuries or assess rehabilitation. However, the widespread use of vision-based systems for this task raises significant privacy concerns, as these systems often capture identifiable facial information. While face blurring is a straightforward method to address privacy, its impact on the accuracy of human pose estimation and, more importantly, the derived kinematic analyses, has remained unclear. This paper aims to quantify this impact, specifically for 3D pose estimation and its downstream applications, to determine if face blurring can be reliably used without compromising the integrity of the analysis.
Methods
The researchers employed a multi-stage approach:
- Face Blurring: They applied Gaussian blur to the facial regions of an existing human gait dataset. A kernel size of 25x25 and a standard deviation of 4.1 were used, determined automatically via OpenCV.
- 3D Human Pose Estimation: They utilized a learnable triangulation algorithm, a standard approach for 3D pose estimation from multi-view images.
- Model Training: Three distinct deep learning models were trained:
- Model #1: Trained and tested on face-blurred images.
- Model #2: Trained on original images and then fine-tuned on face-blurred images.
- Model #3 (Control): Trained and tested on original, unblurred images.
- Kinematic Analysis: After obtaining 3D keypoint coordinates from the pose estimation models, they calculated joint angles for the lower and upper extremities using established methods, with modifications for the human model used.
- Statistical Analysis: To evaluate the effect of face blurring, they performed:
- Mean Per Joint Position Error (MPJPE): To assess landmark localization accuracy.
- One-way repeated measures ANOVA: To statistically determine if face blurring had a significant effect on joint angle calculations.
- Maximum variation (Δ): To quantify the strength of the effect of face blurring on joint angle estimation.
Datasets
- ENSAM Dataset: A multi-view human gait dataset collected in a previous study. This dataset contains 43 subjects (19 females, 24 males) with synchronized video from four cameras and 3D marker data from a VICON system. For this study, the dataset was split into training (27 subjects) and testing (16 subjects) sets. The paper notes that many subjects in the dataset were masked due to the COVID-19 pandemic context.
Key Insights and Ideas
- Privacy vs. Accuracy Trade-off is Manageable: The core finding is that face blurring can effectively protect subject privacy without causing a significant degradation in the accuracy of 3D human pose estimation or the derived kinematic analysis.
- Minimal Impact on Kinematics: While some joint angles showed minor variations, the overall impact of face blurring on kinematic calculations was found to be within acceptable limits (less than 5° difference).
- Direct Training on Blurred Data is Feasible: The study suggests that models can be trained directly on face-blurred datasets, bypassing the need for pre-training on original, unblurred data, which simplifies the workflow for privacy-sensitive applications.
- Head and Neck are Most Affected: Keypoints like the head and neck showed the largest variations in localization error due to face blurring, which is intuitive given their proximity to the blurred region.
Key Results
- Landmark Localization: The MPJPE for experiment #1 (trained on blurred) was comparable to experiment #3 (control, trained on original), with a difference of only 0.3 mm. Experiment #2 (fine-tuned) showed a marginal decrease in MPJPE. The maximum average variation for keypoints was 0.7 mm.
- Joint Angle Significance: A one-way repeated measures ANOVA revealed that only elbow flexion was statistically significantly affected by face blurring (p-value = 0.040), but this effect was described as "relatively weak and acceptable." All other joint angle calculations showed no statistically significant variation.
- Joint Angle Strength: The maximum variation (Δ) in RMSE for joint angles was negligible, generally less than 1°. The largest variation was observed for shoulder flexion (0.6°). Even for elbow flexion, the impact was deemed acceptable.
Limitations and Future Work
- Dataset Specificity: The study was conducted on a gait dataset. The authors acknowledge that other types of movements were not investigated, and it would be beneficial to examine the effect of face blurring on datasets with different motions.
- Subject Masking: A significant portion of the subjects in the dataset were masked due to the COVID-19 pandemic, which might influence the generalizability of the findings.
- Single Camera Setup: The investigation was limited to a single camera setup. Future work could explore the impact of face blurring with different camera configurations.
- Unmasked Subjects: The authors suggest examining unmasked subjects in future research.
- Diverse Datasets: Exploring other datasets beyond gait would provide a more comprehensive understanding of face blurring's effects.
Summary updated