Introduction
Understanding 3D human pose is a cornerstone of computer vision with applications ranging from robotics to surveillance. However, two primary challenges persist: significant viewpoint variation and the complex intrinsic dependencies of the human skeleton. Standard approaches often require extensive labeled data or attempt to discard viewpoint information entirely. This paper proposes a novel unsupervised framework that explicitly disentangles "what the pose is" (view-invariant) from "where it is viewed from" (view-variant).
Methodology
SeBiReNet: A Bio-inspired Architecture
A key contribution of this work is the Sequential Bidirectional Recursive Network (SeBiReNet). Unlike standard Multi-Layer Perceptrons (MLPs) that treat joints as independent points, SeBiReNet leverages the tree-like structure of the human skeleton.
As shown in the figure above, the network consists of two main components:
- Recursive Subnet: Aggregates information from the "fingertips" toward the "root" (pelvis), capturing local-to-global kinematic constraints.
- Diffuse Subnet: Distributes information from the root back to the limbs, ensuring global consistency across the skeleton. This bidirectional flow allows the model to learn complex joint dependencies with a fraction of the parameters required by traditional architectures.
Siamese Denoising Autoencoder and Disentanglement
The overall framework utilizes a Siamese Denoising Autoencoder (DAE) to learn robust representations without manual labels. The model is trained to reconstruct clean 3D poses from corrupted inputs, forcing it to internalize biological constraints like bone length and symmetry.
The encoder maps the input into a latent space split into two distinct features: $h_{vi}$ (view-invariant pose) and $h_v$ (view-variant viewpoint). To enforce this separation, the authors employ a "view-transfer" strategy: swapping viewpoint features between different views of the same pose and requiring the decoder to reconstruct the original configurations. An orthogonal constraint is also applied to $h_v$ to prevent pose information from leaking into the viewpoint representation.
Experiments and Key Results
The framework was evaluated across several major datasets, including Cambridge-Imperial APE, N-UCLA, and NTU RGB+D.
- Unsupervised Action Recognition: On the N-UCLA dataset, the model achieved a state-of-the-art unsupervised accuracy of 80.30%, significantly outperforming previous methods and approaching supervised performance levels.
- Parameter Efficiency: The model is remarkably lightweight. On the NTU RGB+D dataset, it achieved 79.71% accuracy with only 0.27 million parameters, whereas competing state-of-the-art models often require tens of millions.
- Pose Denoising: SeBiReNet demonstrated superior performance in pose reconstruction, reducing the Mean Per Joint Position Error (MPJPE) to 33.39mm, a 50% improvement over standard MLPs.
- Transferability: Representations learned on the APE dataset transferred effectively to N-UCLA and NTU without fine-tuning, indicating the model learns universal human kinematic properties.
Conclusion
By combining a bio-inspired recursive network with a disentangled latent space, this framework provides a highly efficient and effective solution for unsupervised 3D pose representation. Future work may involve integrating temporal dynamics directly into the disentanglement process to further enhance action recognition capabilities.
Summary updated