Human Tracking with Statistical Shape Model and Rough Pose Estimation by Random Forest

Kiyoshi Hashimoto, Hirokatsu Kataoka, Yuji Sato, Masamoto Tanabiki, Yoshimitsu AokiJournal of the Japan Society for Precision Engineering 2015 · DOI

Abstract

Human tracking in surveillance cameras has been a challenging task in the field of computer vision. Tracking objects have large variations such as pose, body shape, clothes, and so on. Especially in parts-based methods, postural change is a big problem since the appearance of a human changes drastically. We deal with this problem by using a statistical shape model for tracking and detection. It represents the variations of postural change and body shape with low dimensions. Our tracking result includes more detailed information on the position and shape of body parts. So we recognize rough pose and body direction to analyze it. These data are useful for security systems or marketing decisions in surveillance.

Introduction

Human tracking in surveillance is notoriously difficult because people are dynamic; they bend, sit, turn, and change shape constantly. Traditional "parts-based" tracking methods often fail when a person's appearance changes drastically due to these postural shifts. Most existing systems either use simple bounding boxes, which lack detail, or complex 3D models, which are computationally expensive. This paper proposes a middle ground: a system that tracks detailed body part positions accurately enough to infer behavior—such as whether a person is looking at a product or has fallen—without requiring massive processing power.

Methodology

The proposed system utilizes a multi-stage approach centered on model integration and efficient searching.

System pipeline from input image to rough pose estimation.

Statistical Shape Model Integration

The core innovation lies in combining two distinct models:

  1. Main-parts Link Model: A skeletal-style model representing the trunk and limbs.
  2. Active Ω Model (AOM): A model focusing on the contour of the head and shoulders.

By applying Canonical Correlation Analysis (CCA), the researchers established a low-dimensional relationship between these models. This allows the system to estimate full body posture even when only the head and shoulders are visible. Tracking occurs in a "latent shape space" rather than raw image space, making it more resilient to noise.

Illustration of (b) the multi-part skeletal link model and (c) the contour-based head-and-shoulder model used for tracking.

Tracking and Estimation

The system employs a Two-Stage Particle Filter to balance speed and accuracy. A "Coarse Search" identifies the general area and rough posture, followed by a "Fine Search" that refines the fitting of individual body parts (head, chest, waist, legs, feet) using a Gaussian distribution to narrow the search space.

Finally, a Random Forest (RF) classifier processes the 13-dimensional state vector from the tracker to categorize the person’s state into three poses (stand, bend, sit) and eight compass directions.

Experimental Results

The system was evaluated using custom annotated video sequences and the standard CAVIAR dataset. Key results include:

  • Tracking Accuracy: The integrated model achieved 81.0% accuracy, outperforming independent models (78.6%).
  • Efficiency: The integrated search was ~43% faster, processing frames in 412.3ms compared to 723.5ms for independent versions.
  • Pose and Direction: The Random Forest achieved 82.7% accuracy for pose classification (standing and sitting were >82%, while "bending" was 72.4%) and 73.7% accuracy for body orientation.

Conclusion and Future Work

By using a statistical model trained on real human movements, the tracker is prevented from "hallucinating" impossible body shapes. While highly effective, the system occasionally struggles with 180-degree orientation symmetry and the ambiguity of the "bending" pose. Future work will focus on refining the likelihood functions in the particle filter to further boost tracking robustness.

Summary updated

Abstract

Human tracking in surveillance camera has been challenging task in the field of computer vision. Tracking objects have large variations such as pose, body shape, clothes and so on. Especially in parts-based methods, postural change is big problem since appearnce of human changes drastically. We deal with this problem to use statistical shape model for tracking and detection. It represents the variations of postural change and body shape with low dimensions. Our trakcing result includes more detailed the position and shape of body parts. So we recognize rough pose and body direction to analyze it. These data is useful for seculity system or marketing decision in surveillance.
BibTeX
@article{hashimoto2015human,
  author = {Kiyoshi Hashimoto and Hirokatsu Kataoka and Yuji Sato and Masamoto Tanabiki and Yoshimitsu Aoki},
  title = {{Human Tracking with Statistical Shape Model and Rough Pose Estimation by Random Forest}},
  journal = {Journal of the Japan Society for Precision Engineering},
  year = {2015},
  doi = {10.2493/jjspe.81.1162},
}
Figure