Abstract
Pose estimation of the human body and hands is a fundamental problem in computer vision, yet learning-based solutions are often constrained by the massive requirement for annotated data. This work presents a framework to improve the efficiency of the data annotation process for 3D pose estimation using Active Learning (AL) in multiview settings. By leveraging multiview geometry, we propose novel AL strategies that identify the most informative frames for human labeling. Furthermore, we incorporate a self-training (ST) loop that utilizes high-confidence pseudo-labels to augment the training set. Our approach demonstrates significant performance gains on the CMU Panoptic Studio and InterHand2.6M benchmarks, reducing annotation costs by up to 80% and turn-around time by 60% while maintaining state-of-the-art accuracy.
Introduction
The advancement of deep learning has significantly improved the accuracy of 3D human pose estimation. However, these models require vast amounts of high-quality labeled data. In multiview environments, where dozens of cameras capture a single scene, the annotation process is particularly grueling. A human annotator may spend a full minute labeling just one frame of 3D motion across multiple views.
The core challenge is efficiency: not all captured frames are equally informative. Redundant frames, such as a subject standing still, contribute little to model improvement. This paper addresses the bottleneck by introducing an Active Learning framework tailored for the geometric complexities of 3D pose estimation.
The Proposed Framework
The system architecture is designed to minimize human intervention by intelligently partitioning the data pool into frames requiring human expertise and those that can be handled automatically.
As shown in the figure above, the framework begins with an unlabeled pool of gathered raw data. A trained model performs inference to generate initial pseudo-labels. The Active Learning (AL) and Self-Training (ST) module then evaluates these labels. "Bad" or uncertain pseudo-labels are sent for human annotation, while "good" or high-confidence pseudo-labels are used for data augmentation. Both sets of labels eventually populate the labeled pool used to retrain the pose estimation model in an iterative loop.
Active Learning and Self-Training Strategies
The authors propose two primary AL strategies that utilize multiview geometry:
- CoreSet-Poses (Diversity-based): This strategy ensures the labeled dataset covers a wide range of the "pose space." By calculating geometric distances between 3D poses, the system selects a diverse set of actions (e.g., sitting, jumping, reaching) to avoid redundant labeling.
- Multi-View Consistency (OURS-MC): This uncertainty-based approach uses triangulation error as a proxy for model confidence. If 2D predictions from different cameras do not converge to a single 3D point, the model is deemed uncertain, and the frame is flagged for human annotation.
To further boost efficiency, the Self-Training (ST) component identifies frames where the model is exceptionally confident (low triangulation error). These frames receive automated "pseudo-labels," which are added to the training set without human cost. To prevent "model drift," a strict inlier requirement ensures that only highly consistent multiview predictions are used.
Experimental Results
The framework was evaluated on the CMU Panoptic Studio (body pose) and InterHand2.6M (hand pose) datasets. The results indicate that the proposed multiview strategies significantly outperform standard single-view AL techniques, such as Maximum Peak Entropy (MPE).
As illustrated in the performance comparison above, the AL-based annotation process reaches a 3D Mean Keypoint Prediction Error (MKPE) of approximately 2.7mm in just 20 hours, whereas the conventional annotation process requires 50 hours to achieve the same level of precision. This represents a 60% reduction in turn-around time. Furthermore, the system achieved an 80% reduction in total annotation cost on the CMU Panoptic benchmark.
Conclusion
By rethinking the annotation process through the lens of multiview geometry, this work demonstrates that significant efficiency gains are possible in 3D pose estimation. The combination of uncertainty-based Active Learning and high-confidence Self-Training allows for the creation of high-performance models with a fraction of the human effort typically required. Future work may involve incorporating temporal information from video sequences to further reduce data redundancy and exploring applications in uncalibrated, "in-the-wild" camera setups.
Summary updated