Abstract
Estimating 3D human poses from a single 2D camera view is a classic "ill-posed" problem due to depth ambiguity and self-occlusion. This paper proposes a novel deep learning architecture that addresses temporal redundancy and the loss of subtle joint movements through multi-joint feature fusion. By integrating a multi-branch temporal convolution module for motion extraction, a joint feature fusion module for inter-limb relationships, and a joint restraint module for spatial consistency, the network achieves significant accuracy gains. On the Human3.6M dataset, the method achieves a Mean Per Joint Position Error (MPJPE) of 29.0 mm, representing a 4.9% improvement over state-of-the-art baselines, with even higher performance in complex action sequences.
Introduction
Monocular 3D human pose estimation (HPE) is critical for applications in augmented reality, sports analysis, and human-computer interaction. However, current models often struggle with two main issues: temporal redundancy, where adjacent video frames provide overlapping information that can "wash out" details, and the failure to capture minute joint movements. This research introduces a three-pronged architecture designed to rethink how joint features are fused across time and space to produce more anatomically grounded 3D poses.
Methodology
The proposed network architecture is built around three core modules that work in tandem to refine the 3D estimation process:
- Multi-Branch Joint Motion Feature Extraction: Instead of a single path, the network uses four parallel branches with strided convolutions at different intervals. This allows the model to capture motion at multiple temporal scales, filtering out redundancy while highlighting subtle changes like wrist flicks or balance shifts.
- Joint Feature Fusion Module: This module models the high-dimensional relationships between different body parts. By grouping joints into five topological categories (torso, left/right arms, and left/right legs), the network learns how the movement of one limb relates to another, ensuring cohesive body motion.
- Joint Restraint Module: Acting as a "sanity check," this module uses the 2D joint positions from the middle frame as an implicit constraint. By fusing this 2D evidence with the initial 3D estimate, the network prunes unrealistic 3D poses that do not align with the observed 2D image.
A key insight of this approach is that topology matters. By grouping joints based on the human skeleton's actual structure rather than treating them as independent points, the model learns more meaningful spatial relationships.
Results and Discussion
The model was rigorously evaluated on three gold-standard datasets: Human3.6M, MPI-INF-3DHP, and HumanEva.
- Human3.6M: The network achieved an MPJPE of 29.0 mm. The improvement was most pronounced in difficult scenarios; for "Sitting Down," the error dropped by 7.7%, and for "Walking Dog," it dropped by 8.2%.
- MPI-INF-3DHP: The model showed exceptional generalization, with a 36.2% reduction in MPJPE and a 12.9% increase in the Area Under the Curve (AUC) metric.
- Efficiency: Despite the multi-branch approach, the model maintains a competitive parameter count of approximately 30M, which is lower than several heavy-duty transformer-based models like MixSTE.
Conclusion
By effectively fusing multi-scale temporal features and enforcing 2D spatial constraints, the proposed network significantly improves the accuracy and robustness of monocular 3D human pose estimation. Future work will focus on model compression and reducing input sequence lengths to enable real-time applications in latency-sensitive environments.
Summary updated