Human Pose Estimation from Silhouettes - A Consistent Approach Using Distance Level Sets

Cristian Sminchisescu, Alexandru TeleaWSCG 2002 · OpenReview

Introduction

Estimating 3D human pose from 2D image silhouettes is a foundational yet challenging problem in computer vision. It is inherently "ill-posed" because a single 2D silhouette can correspond to numerous 3D configurations. This ambiguity often results in a mathematical cost surface—the landscape searched to find the best fit—that is jagged and riddled with "spurious minima" or false peaks. Traditional methods frequently fail by settling on these incorrect poses.

This paper addresses these challenges by introducing a robust, continuous likelihood measure. The core innovation is the Consistency Principle: a successful model fit should not only be "in bounds" of a silhouette but must also "explain" the entire silhouette area. By combining attraction forces with area-based explanation, the authors create a smoother cost surface that significantly improves tracking stability.

Methodology: The Dual-Component Likelihood

The authors represent the human body using an articulated 3D model composed of a kinematic skeleton covered by "flesh" made of superquadric ellipsoids. To align this model with image data, they propose a likelihood function with two distinct components:

1. Silhouette Attraction via Distance Level Sets

To guide the model toward the silhouette, the authors employ Distance Level Sets calculated via the Fast Marching Method (FMM). This technique creates a potential field that effectively "pulls" the projected model nodes toward the interior of the image silhouette. Unlike discrete methods, this approach provides a continuous gradient, allowing for the use of efficient second-order optimization techniques like Levenberg-Marquardt.

Figure 3: The silhouette processing pipeline: (a) original image, (b) extracted binary silhouette, (c) distance map generated via FMM, and (d) the resulting level sets used for the attraction term.

2. Area Overlap for Global Consistency

The "Area Overlap" term serves as the explanation component. It calculates the difference between the area of the target silhouette and the area covered by the projected model. This prevents the optimization from settling on "consistent but incorrect" poses—such as a tiny model or a poorly positioned limb that technically sits inside the silhouette but fails to match the person's actual volume.

Robustness through Skeleton-based Smoothing

Real-world silhouettes are often noisy and jagged, which can confuse high-fidelity models. To mitigate this, the authors introduce Skeleton-based Smoothing. They use FMM to extract the "medial axis" (skeleton) of the raw silhouette, prune away small noisy branches, and then "re-inflate" the skeleton. This process creates a smoothed, idealized version of the silhouette that preserves core topology and orientation while being easier for the superquadric model to approximate.

Experimental Results and Insights

The approach was evaluated on tracking sequences of humans performing various movements, including arm waving and walking. The combination of attraction and area-overlap terms creates a much wider "basin of attraction," making the system less sensitive to initial conditions.

Figure 5: Qualitative results demonstrating successful 3D pose estimation on real-world sequences. Rows (a-d) and (e-h) show the model accurately tracking arm waving and walking motions respectively.

Key Findings:

  • Efficiency: The distance transform and smoothing calculations are highly efficient, enabling near-real-time performance on contemporary hardware.
  • Stability: The continuous formulation leads to faster and more stable convergence compared to purely discrete silhouette matching methods.
  • Consistency: By requiring the model to explain the silhouette area, the authors eliminate a whole family of incorrect poses that would otherwise appear optimal to simpler algorithms.

Conclusion and Future Work

While the method is robust, the authors acknowledge that single-view 3D estimation remains inherently ambiguous (e.g., determining if an arm points forward or backward). Future work suggests that temporal constraints or multi-camera setups are necessary for full disambiguation. Additionally, the authors propose exploring direct skeleton matching, where extracted silhouette skeletons are used as primary features for the fitting process rather than just for smoothing.

Summary updated

BibTeX
@inproceedings{sminchisescu2002human,
  author = {Cristian Sminchisescu and Alexandru Telea},
  title = {{Human Pose Estimation from Silhouettes - A Consistent Approach Using Distance Level Sets}},
  booktitle = {WSCG},
  year = {2002},
  url = {https://openreview.net/forum?id=9RIM9G852tj},
}
Figure