3D Pictorial Structures for Multiple Human Pose Estimation

Vasileios Belagiannis, Sikandar Amin, Mykhaylo Andriluka, Bernt Schiele, Nassir Navab, Slobodan Ilic

Abstract

This paper introduces a 3D Pictorial Structures (3DPS) model for the simultaneous 3D pose estimation of multiple people from multiple camera views. By formulating the problem as a collision-aware Conditional Random Field (CRF) and discretizing the search space through multi-view triangulation, the method effectively eliminates "ghost" limb hypotheses without requiring prior knowledge of person identities.

Introduction

Estimating the 3D pose of multiple people from multiple camera views is a significant challenge in computer vision. The complexity arises from the high-dimensional state space of human limbs and the difficulty of data association across different viewpoints. Without pre-defined identities, traditional triangulation often results in "ghost" limbs—false positives created by mismatched 2D detections. This paper addresses these challenges by moving away from continuous volumetric searches toward a bottom-up, discrete state space approach.

Methodology: 3D Pictorial Structures (3DPS)

The proposed 3DPS framework utilizes a Conditional Random Field (CRF) where the human body is represented by 11 discrete variables corresponding to major limbs and parts.

The 3D Pictorial Structures model representing the human body with 11 discrete parts (Y1-Y11) and their kinematic connections.

Instead of searching a massive continuous 3D volume, the method starts with standard 2D part detectors in every camera view. It then triangulates every possible pair of 2D joints across views to create a set of 3D joint hypotheses. This drastically reduces the search space to a few hundred candidate points. The CRF incorporates several key potentials to find the most likely configuration:

  • Reprojection Error: Ensures 3D hypotheses align with 2D detections.
  • Multi-view Visibility: Penalizes hypotheses not visible in a sufficient number of cameras.
  • Part Length: Anatomical priors to ensure realistic limb proportions.
  • Collision Potential: This "secret sauce" penalizes symmetric parts from occupying the same physical 3D space, preventing the model from merging different people into a single skeleton.

Inference is performed using Loopy Belief Propagation, allowing the model to efficiently assemble skeletons from the candidate 3D points.

Experimental Setup and Results

The model was evaluated on single-person datasets (HumanEva-I, KTH Multiview Football II) and complex multi-person benchmarks introduced by the authors: Campus and Shelf. The "Shelf" dataset is particularly challenging, featuring four people disassembling a shelf with significant self-occlusion and overlapping bodies.

Qualitative results showing consistent 3D pose estimation across multiple camera views in a multi-person scenario.

On the Shelf dataset, the model achieved a 71.3% Percentage of Correct Parts (PCP). Notably, the system maintained high accuracy without requiring explicit person tracking or identity information, proving the robustness of the geometric and anatomical constraints. The collision potential effectively resolved ambiguities even when individuals were in close proximity.

Conclusion

The 3DPS framework provides a robust, identity-agnostic solution for multi-person 3D pose estimation. By combining geometric constraints with anatomical priors in a discretized state space, the method achieves competitive accuracy and operational efficiency, running at approximately 1 frame per second on a standard laptop. While dependent on the quality of initial 2D detections, the model's reliance on 3D geometric rigidity makes it highly effective for complex, multi-view scenarios.

Summary updated

Abstract

In this work, we address the problem of 3D pose estimation of multiple humans from multiple views. This is a more challenging problem than single human 3D pose estimation due to the much larger state space, partial occlusions as well as across view ambiguities when not knowing the identity of the humans in advance. To address these problems, we first create a reduced state space by triangulation of corresponding body joints obtained from part detectors in pairs of camera views. In order to resolve the ambiguities of wrong and mixed body parts of multiple humans after triangulation and also those coming from false positive body part detections, we introduce a novel 3D pictorial structures (3DPS) model. Our model infers 3D human body configurations from our reduced state space. The 3DPS model is generic and applicable to both single and multiple human pose estimation. In order to compare to the state-of-the art, we first evaluate our method on single human 3D pose estimation on HumanEva-I [22] and KTH Multiview Football Dataset II [8] datasets. Then, we introduce and evaluate our method on two datasets for multiple human 3D pose estimation. In order to compare to the state-of-the art, we first evaluate our method on single human 3D pose estimation on HumanEva-I [22] and KTH Multiview Football Dataset II [8] datasets. Then, we introduce and evaluate our method on two datasets for multiple human 3D pose estimation.
BibTeX
@inproceedings{belagiannis2014d,
  author = {Vasileios Belagiannis and Sikandar Amin and Mykhaylo Andriluka and Bernt Schiele and Nassir Navab and Slobodan Ilic},
  title = {{3D Pictorial Structures for Multiple Human Pose Estimation}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2014},
  doi = {10.1109/cvpr.2014.216},
}
Figure