Joint Representation of Multiple Geometric Priors via a Shape Decomposition Model for Single Monocular 3D Pose Estimation

Mengxi Jiang, Zhuliang Yu, Cuihua Li, Yunqi LeiarXiv 2019 · arXiv

Abstract

Estimating 3D human poses from single 2D images is a challenging ill-posed problem due to depth ambiguity. This paper proposes a Shape Decomposition Model (SDM) that recovers 3D poses by splitting them into two components: a global structure and local deformations. By employing a hybrid representation—sparse for global shapes and dense for local details—the model effectively generalizes to complex poses even with limited training data. Experimental results on benchmarks like Human3.6M and CMU Mocap demonstrate significant improvements over existing unsupervised methods, particularly in handling highly deformed poses.

Introduction

The recovery of 3D human pose from a single 2D image is a fundamental task in computer vision. However, it is inherently ill-posed because a single 2D projection can correspond to multiple 3D configurations, a phenomenon known as depth ambiguity.

The challenge of 3D pose estimation from a single monocular camera view, illustrating the projection from 3D space to a 2D image plane.

While deep learning approaches have shown promise, they often require vast amounts of annotated 3D data. Traditional Sparse Representation (SR) models, which rely on a dictionary of basis poses, struggle to generalize when the target pose (e.g., sitting) is not well-represented in the training set (e.g., walking). This paper addresses these limitations by introducing a "divide and conquer" strategy for pose representation.

Methodology: Shape Decomposition Model (SDM)

The core innovation is the decomposition of a 3D pose $Y$ into two components: a global structure $U$ and local deformations $V$: $$Y = U + V$$

Overview of the Shape Decomposition Model (SDM) pipeline, illustrating the separation of global structure and local deformation bases for 3D pose reconstruction.

Hybrid Representation Strategy

The model treats these components with different mathematical constraints:

  1. Global Structure ($U$): Modeled using Sparse Representation. It assumes that the general human skeleton lies in a low-dimensional subspace, requiring only a few basis poses from a learned dictionary to describe the coarse shape.
  2. Local Deformations ($V$): Modeled using a Dense (Collaborative) Representation. The authors argue that while global poses vary greatly, local joint configurations (like a knee bend) are often shared across different activities. A dense representation allows the model to "borrow" these local details from the entire dictionary to reconstruct novel, complex poses.

Joint Dictionary Learning and Optimization

The framework learns two dictionaries simultaneously—one for coarse global shapes and another for fine-grained details. Reconstruction is achieved using the Alternating Direction Method of Multipliers (ADMM), which iteratively optimizes the rotation matrix and combination coefficients to minimize the distance between the projected 3D pose and the observed 2D joints.

Experiments and Results

The SDM was evaluated on several benchmarks, including Human3.6M, HumanEva-I, CMU Mocap, and MPII Human Pose.

Performance on Complex Poses

The model showed significant improvements over unsupervised baselines. On Human3.6M, it achieved a 12% overall accuracy improvement. Notably, for highly deformed categories like "Sitting," the error was reduced by 66% compared to previous state-of-the-art unsupervised models. This confirms the effectiveness of the dense representation for local deformations.

Robustness and Generalization

Tests on the CMU Mocap dataset demonstrated the model's ability to generalize to diverse motions such as "Dancing" and "Hoop." Furthermore, the approach proved robust to Gaussian noise in 2D joint inputs, maintaining high reconstruction quality even with imperfect data.

Discussion and Conclusion

The SDM leverages the "local universality" of human joint movements to overcome the limitations of standard sparse representation in 3D pose estimation. By separating global structure from local detail, the model achieves a coarse-to-fine estimation that is both robust and data-efficient. While the model excels at complex deformations, it occasionally overfits on simpler, linear motions like walking. Future work could extend this decomposition framework to other non-rigid objects, such as human hands or faces.

Summary updated

BibTeX
@misc{jiang2019joint,
  author = {Mengxi Jiang and Zhuliang Yu and Cuihua Li and Yunqi Lei},
  title = {{Joint Representation of Multiple Geometric Priors via a Shape Decomposition Model for Single Monocular 3D Pose Estimation}},
  year = {2019},
  eprint = {1905.13466},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1905.13466},
}
Figure