Abstract
Estimating 3D human pose from a single RGB image is a challenging task due to depth ambiguity and frequent self-occlusions. This paper introduces a Feature Boosting Network designed to exploit spatial dependencies between body joints. By integrating a Graphical ConvLSTM within a stacked Hourglass architecture, the model propagates information across a skeletal graph. Furthermore, a Context Consistency Gate (CCG) is employed to filter out unreliable local features caused by noise or occlusion. Experimental results on major benchmarks demonstrate that this approach significantly improves 3D pose estimation accuracy for both hands and full bodies.
Introduction
3D pose estimation from a single 2D image is a classic "ill-posed" problem in computer vision. Without depth information, networks must infer the third dimension from visual context and anatomical constraints. Standard convolutional neural networks (CNNs) often struggle when body parts are occluded or when articulations are complex.
In "Feature Boosting Network For 3D Pose Estimation," Liu et al. propose that anatomical relationships should be integrated directly into the feature-learning process rather than applied as a post-processing step. Their architecture "boosts" intermediate features by modeling the complex graphical relationships between joints, such as the physical connection between a wrist and an elbow or the symmetrical correlation between left and right knees.
Methodology
The core of the framework is the Long Short-Term Dependence-aware (LSTD) module, which is embedded within a stacked Hourglass CNN architecture. This module consists of two primary components: the Graphical ConvLSTM and the Context Consistency Gate (CCG).
Graphical ConvLSTM
Unlike standard LSTMs that process data in a linear sequence, the Graphical ConvLSTM links units according to the actual topology of the human skeleton. This allows the network to propagate information across the body's graph, enabling it to estimate the position of occluded joints based on the known positions of neighboring joints. The graph accounts for both physical connections and symmetrical relations.
Context Consistency Gate (CCG)
To handle noisy or occluded features, the CCG acts as a trust mechanism. It predicts expected joint features based on global context and compares them to local convolutional evidence. If the local features are unreliable (e.g., a hand hidden behind the torso), the gate suppresses them and relies more on the context-based prediction to "boost" the representation.
Experiments and Results
The authors evaluated the model on several rigorous benchmarks, including the 3DHandPose dataset and Human3.6M (the gold standard for 3D body pose).
On the 3DHandPose dataset, the model achieved a Percentage of Correct Keypoints (PCK) of 89.5% at a 20mm threshold, a significant improvement over the baseline. On Human3.6M, the model achieved a mean Euclidean distance error of 61.10 mm, outperforming several competitive state-of-the-art models.
Key Insights
- Symmetry Matters: Modeling symmetrical connections (e.g., left hand to right hand) significantly improves accuracy, as their motions are often highly correlated.
- Feature-Level Boosting: Refining intermediate features rather than output coordinates allows both 2D heatmap generation and 3D depth regression to benefit from cleaner data.
- Context as a Filter: The CCG effectively turns the network into a self-correcting system by comparing local evidence against global context.
Conclusion
The Feature Boosting Network demonstrates that explicitly modeling graphical dependencies and using context to gate unreliable features can significantly advance 3D pose estimation. While hardware memory limits currently restrict the number of stacked sub-networks, the approach provides a robust foundation for future work in "in-the-wild" pose estimation and dynamic action-based skeletal graphs.
Summary updated