3D Pictorial Structures for Multiple View Articulated Pose Estimation

Magnus Burenius, Josephine Sullivan, Stefan CarlssonCVPR 2013 · DOI

Introduction

Estimating the 3D pose of a human—determining the precise positions and orientations of all limbs—remains a significant challenge in computer vision due to the high-dimensional nature of the search space. While 2D "pictorial structures" have long been a standard for efficient pose estimation by modeling body parts as nodes in a tree graph, extending this framework to 3D introduces exponential complexity. A 2D pose typically involves a 3D state (2D translation and 1D rotation), whereas a 3D pose requires a 6D state per part (3D translation and 3D rotation).

The proposed framework estimates 3D pose by aggregating evidence from multiple calibrated camera views.

Previous 3D pose estimation methods often relied on stochastic searches that lacked global optimality guarantees or attempted to "lift" 2D detections into 3D, which frequently resulted in global inconsistencies. This research investigates whether 3D space can be discretized finely enough to find a globally optimal solution using dynamic programming without exceeding computational limits.

The 3D Pictorial Structures Model

The authors propose a 3D Pictorial Structures framework that models the human body as a kinematic tree (e.g., Pelvis → Torso → Arms). This tree-based representation is crucial because it allows the "max-product" algorithm to solve for the optimal pose in linear time relative to the number of body parts.

The kinematic tree model represents body parts as nodes, enabling efficient global optimization via dynamic programming.

To make the 6D search space tractable, the model employs specific discretization strategies:

  • Translation: A 3D grid is established for spatial positioning.
  • Rotation: An evenly spread set of 3D rotations (SO(3)) is generated using unit quaternions and "best-candidate sampling."
  • Constraints: The model incorporates fixed limb lengths, hard joint angle limits (e.g., preventing unnatural elbow bends), and view constraints that measure how well a 3D part's projection aligns with image data across all camera views.

One common issue with tree models is "double-counting," where multiple limbs are assigned to the same high-scoring spatial region. The authors address this with a two-step heuristic: they first find the best pose, then fix one limb and re-optimize for the other while enforcing non-intersection constraints using capsule-based part modeling.

Experimental Evaluation

The model was evaluated using a professional soccer dataset consisting of 214 consecutive frames from three calibrated cameras. Ground truth was established via manual 2D annotation and affine factorization.

Key findings from the experiments include:

  • Tractability: A $32^3$ translation grid combined with an $8^3$ rotation grid (approximately 32,000 translations and 512 rotations) provides sufficient resolution for accurate estimation while remaining computationally feasible on standard workstations.
  • Inference Speed: Basic inference using view and skeletal constraints takes approximately 1 second per frame. However, incorporating complex joint angle priors increases this to about 5 minutes, highlighting a significant computational trade-off.
  • Robustness: The 3D framework significantly outperformed standalone 2D part detectors by effectively aggregating evidence across multiple views and enforcing global skeletal consistency.

Conclusion and Future Work

This work demonstrates that finding globally optimal 3D human poses is tractable through careful discretization and the use of pictorial structures. While the current model relies on fixed limb lengths and basic HOG-based detectors, the framework is designed to be modular. Future improvements could involve integrating modern deep-learning-based detectors, implementing "coarse-to-fine" search strategies to reduce computational overhead, and developing more integrated methods for handling self-intersections without sacrificing the efficiency of the tree-search algorithm.

Summary updated

BibTeX
@inproceedings{burenius2013d,
  author = {Magnus Burenius and Josephine Sullivan and Stefan Carlsson},
  title = {{3D Pictorial Structures for Multiple View Articulated Pose Estimation}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2013},
  doi = {10.1109/cvpr.2013.464},
}
Figure