Abstract
Estimating 3D human poses from 2D images or video sequences is a challenging "ill-posed" problem due to depth ambiguity. This paper presents DC-GCT (Double-chain Graph Convolutional Transformer), a novel architecture that simultaneously models local skeletal constraints and global body dependencies. By utilizing a parallel "double-chain" structure—comprising Local-to-Global and Global-to-Local pathways—the model achieves state-of-the-art accuracy on major benchmarks like Human3.6M and MPI-INF-3DHP. Notably, DC-GCT maintains high computational efficiency, making it suitable for real-time video applications.
Introduction
The process of "2D-to-3D lifting" involves reconstructing a 3D skeleton from 2D joint coordinates. Historically, researchers have relied on two primary methods: Graph Convolutional Networks (GCNs), which excel at modeling local physical constraints (e.g., limb proportions), and Transformers, which capture global dependencies (e.g., cross-body coordination). However, GCNs often fail to grasp long-range relationships, while Transformers can overlook the inherent structural logic of the human skeleton.
Furthermore, many existing multi-frame models for video are computationally intensive, hindering their deployment in real-time scenarios such as Augmented Reality (AR) or human-computer interaction.
Methodology: The DC-GCT Architecture
The core innovation of this work is the Double-chain Constraint mechanism. Unlike sequential models, DC-GCT processes features through two parallel pipelines to capture a multi-level representation of the human body:
- Local-to-Global Chain: This path utilizes a Local Constraint Module (LCM) based on GCNs to model immediate joint neighbors, followed by a Global Constraint Module (GCM) to integrate broader body context.
- Global-to-Local Chain: This inverse path ensures that global positioning informs the refinement of local joint details.
To synchronize these chains, the authors developed a Feature Interaction Module (FIM) that fuses and maps features between the two paths, preventing information loss during the transition between local and global scales. For video data, a Temporal Embedding method is introduced that "guides" the embedding of a sequence using joint data from the target frame, providing temporal context with minimal computational overhead.
Experimental Results
The model was rigorously evaluated on the Human3.6M and MPI-INF-3DHP datasets. Key findings include:
- State-of-the-Art Accuracy: On Human3.6M, DC-GCT achieved a Mean Per Joint Position Error (MPJPE) of 48.4 mm using detected 2D poses, outperforming previous methods across all action categories. With ground truth 2D locations, the error dropped to 32.4 mm.
- Efficiency: DC-GCT is significantly more lightweight than its predecessors. It utilizes approximately 2.3M parameters, compared to 9.6M for Poseformer and 18.9M for other leading models, while delivering superior performance.
- Temporal Consistency: The model maintains high accuracy in video sequences with a negligible increase in FLOPs compared to single-frame models.
Conclusion and Future Work
DC-GCT demonstrates that parallelizing local and global constraint modeling is highly effective for 3D pose estimation. While the model sets new benchmarks, challenges remain in scenarios involving heavy occlusion or extreme poses that deviate significantly from training data. Future research will focus on enhancing the architecture's robustness to "in-the-wild" conditions and improving handling for partial body detections.
Summary updated