Abstract
Estimating 3D human poses from 2D landmarks without 3D supervision is a challenging, ill-posed problem due to depth ambiguity. We propose the Invariant Teacher and Equivariant Student (ITES) framework to address the "regularization-flexibility dilemma." ITES utilizes a heavily regularized, rotation-invariant teacher network to establish physically plausible 3D pose priors and a flexible, rotation-equivariant student network to refine joint depth. Our method achieves state-of-the-art unsupervised performance on the Human3.6M dataset, reducing the Procrustes-aligned mean per-joint position error (P-MPJPE) by 11.4% and outperforming several weakly-supervised approaches.
Introduction
3D human pose estimation is a cornerstone for applications in animation, sports analytics, and human-computer interaction. While supervised methods yield high accuracy, they rely on massive datasets of paired 2D-3D annotations, which are difficult to acquire. Unsupervised learning offers a scalable alternative but suffers from the inherent ambiguity of perspective projection—where multiple 3D configurations can map to the same 2D pose.
Existing unsupervised methods often rely on strong priors to ensure physical plausibility. However, this creates a trade-off: excessive regularization leads to underfitting (inability to capture diverse poses), while insufficient regularization leads to overfitting (physically impossible skeletons). ITES resolves this by decoupling the estimation process into two specialized components.
Methodology
The ITES framework consists of a two-stage Teacher-Student distillation process designed to balance structural constraints with representational flexibility.
The Invariant Teacher Network
The teacher network focuses on strict regularization. It employs a "pose-dictionary" approach, representing any 3D pose as a linear combination of learned "pose atoms." This constraint ensures that the output remains within the space of physically plausible human shapes. To resolve viewpoint ambiguity, the teacher is trained with a cycle-consistent loss that enforces 3D rotation-invariance. Regardless of the input 2D view, the teacher is forced to predict the same canonical 3D pose.
The Equivariant Student Network
The student network provides the necessary flexibility. It utilizes an Adaptive Graph Convolutional Network (AGCN) that models both physical bone connections and non-physical joint correlations (e.g., hand-to-foot coordination). Unlike the teacher, the student is trained to be 3D rotation-equivariant, meaning its 3D output should rotate in accordance with changes in the input 2D view. The student learns via knowledge distillation from the teacher, refining the coarse canonical estimates into view-specific 3D poses.
Experiments and Results
The model was primarily trained and evaluated on the Human3.6M dataset. To test generalization, the authors evaluated the model on the MPI-INF-3DHP dataset without further fine-tuning.
Quantitative Performance
ITES achieved a new state-of-the-art for unsupervised 3D pose estimation. On Human3.6M, the P-MPJPE was reduced from 58.0mm to 51.9mm, an 11.4% improvement. Notably, ITES outperformed weakly-supervised methods like RepNet and AIGN, which utilize additional information such as unpaired 3D poses or multi-view data during training.
Ablation Studies
Ablation results highlighted the critical role of the rotation-invariant loss. Removing this loss from the teacher caused the Mean Per-Joint Position Error (MPJPE) to increase from 85.6mm to 153.5mm, confirming that invariance is the key mechanism for solving projection ambiguity.
Conclusion
ITES demonstrates that decoupling regularization and flexibility through a teacher-student framework is highly effective for unsupervised 3D pose estimation. By combining rotation-invariance for stability and rotation-equivariance for detail, the model overcomes the limitations of single-network architectures. Future work includes incorporating temporal dynamics from video sequences to further resolve depth ambiguities and extending the framework to multi-person scenarios.
Summary updated