Abstract
This paper introduces a 3D Pictorial Structures (3DPS) model for the simultaneous 3D pose estimation of multiple people from multiple camera views. By formulating the problem as a collision-aware Conditional Random Field (CRF) and discretizing the search space through multi-view triangulation, the method effectively eliminates "ghost" limb hypotheses without requiring prior knowledge of person identities.
Introduction
Estimating the 3D pose of multiple people from multiple camera views is a significant challenge in computer vision. The complexity arises from the high-dimensional state space of human limbs and the difficulty of data association across different viewpoints. Without pre-defined identities, traditional triangulation often results in "ghost" limbs—false positives created by mismatched 2D detections. This paper addresses these challenges by moving away from continuous volumetric searches toward a bottom-up, discrete state space approach.
Methodology: 3D Pictorial Structures (3DPS)
The proposed 3DPS framework utilizes a Conditional Random Field (CRF) where the human body is represented by 11 discrete variables corresponding to major limbs and parts.
Instead of searching a massive continuous 3D volume, the method starts with standard 2D part detectors in every camera view. It then triangulates every possible pair of 2D joints across views to create a set of 3D joint hypotheses. This drastically reduces the search space to a few hundred candidate points. The CRF incorporates several key potentials to find the most likely configuration:
- Reprojection Error: Ensures 3D hypotheses align with 2D detections.
- Multi-view Visibility: Penalizes hypotheses not visible in a sufficient number of cameras.
- Part Length: Anatomical priors to ensure realistic limb proportions.
- Collision Potential: This "secret sauce" penalizes symmetric parts from occupying the same physical 3D space, preventing the model from merging different people into a single skeleton.
Inference is performed using Loopy Belief Propagation, allowing the model to efficiently assemble skeletons from the candidate 3D points.
Experimental Setup and Results
The model was evaluated on single-person datasets (HumanEva-I, KTH Multiview Football II) and complex multi-person benchmarks introduced by the authors: Campus and Shelf. The "Shelf" dataset is particularly challenging, featuring four people disassembling a shelf with significant self-occlusion and overlapping bodies.
On the Shelf dataset, the model achieved a 71.3% Percentage of Correct Parts (PCP). Notably, the system maintained high accuracy without requiring explicit person tracking or identity information, proving the robustness of the geometric and anatomical constraints. The collision potential effectively resolved ambiguities even when individuals were in close proximity.
Conclusion
The 3DPS framework provides a robust, identity-agnostic solution for multi-person 3D pose estimation. By combining geometric constraints with anatomical priors in a discretized state space, the method achieves competitive accuracy and operational efficiency, running at approximately 1 frame per second on a standard laptop. While dependent on the quality of initial 2D detections, the model's reliance on 3D geometric rigidity makes it highly effective for complex, multi-view scenarios.
Summary updated