Learning articulated body models for people re-identification

Davide Baltieri, Roberto Vezzani, Rita CucchiaraACM Multimedia 2013 · DOI

TL;DR

This paper introduces a novel approach to people re-identification by leveraging articulated 3D body models derived from RGB-D sensors. By mapping appearance features onto a learned, segmented skeleton, they achieve a view-invariant signature that significantly outperforms existing 2D and non-articulated 3D methods.

What Problem Does This Paper Address?

People re-identification (re-ID) is crucial for surveillance and forensics, aiming to match individuals across different cameras and times. Traditional methods rely on 2D appearance features (color, texture, shape), which are inherently limited by viewpoint changes, occlusions, and low discriminability. While 3D models have been explored, rigid 3D models struggle with the natural articulation of the human body, and existing articulated motion capture techniques haven't been applied to re-ID. This paper tackles the challenge of creating robust, view-invariant person signatures by effectively utilizing articulated 3D body models.

Methods

The core of this work lies in mapping appearance features from RGB-D data onto an articulated 3D body model. Here's a breakdown of their key methods:

  • RGB-D Data and Skeleton Extraction: They utilize data from Microsoft Kinect sensors, specifically the color, depth, and skeleton streams provided by OpenNi.
  • Articulated Body Model: Instead of a fixed skeleton, they learn a "bone" set by recursively splitting the initial OpenNi skeleton. This process aims to optimize the skeleton's segmentation for re-identification.
  • Bone-wise Color Descriptor: Local color descriptors (RGB histograms with 8-bin quantization) are extracted from the color image and assigned to specific bones of the articulated model.
  • PCA for Dimensionality Reduction: The concatenated bone histograms form a high-dimensional signature, which is then reduced using Principal Component Analysis (PCA) to create a more manageable feature vector.
  • Relaxed Pairwise Metric Learning (RPML): To compare person signatures, they employ RPML, a learned metric that simultaneously performs feature selection and assigns weights to different body parts, enhancing discriminative power.

Datasets

  • Custom Kinect Dataset: The authors created a new dataset using Microsoft Kinect. It comprises 450 shots of 40 individuals, including relative skeleton and point cloud data. This dataset was split into training (for metric learning and bone fragmentation) and testing sets.

Key Insights and Ideas

  • Articulated Models are Key: The paper strongly suggests that exploiting the natural articulation of the human body is a significant step forward for re-ID, moving beyond rigid 3D models or purely 2D approaches.
  • Learned Skeleton Segmentation: The idea of automatically fragmenting the skeleton to create "bones" that are more semantically relevant for re-identification is a novel contribution. This allows for better spatial localization of appearance features.
  • View-Invariant Signatures: By mapping features onto an articulated model, the resulting signature becomes more invariant to changes in viewpoint, a major hurdle in traditional re-ID.
  • Metric Learning for Body Parts: The use of RPML to learn weights for different body parts is crucial. It implies that some body parts (or their learned fragments) are more discriminative for re-identification than others.

Key Results

  • Significant Performance Improvement: The proposed method, combining the learned metric and bone fragmentation, significantly outperforms both the base system (using OpenNi skeleton and uniform weights) and state-of-the-art methods like Sarc3D (non-articulated 3D) and SDALF (2D).
  • Metric Learning Dominance: Figure 5(a) shows that the learned metric provides a substantial boost in performance, even with the base skeleton.
  • Bone Fragmentation Adds Value: Further improvements are seen when the automatic bone fragmentation is also incorporated.
  • Robustness to Pose Changes: Qualitative results (Figure 4) demonstrate that the proposed method is much more resilient to pose variations compared to Sarc3D and SDALF.
  • Efficiency: The creation of a person model is fast (<10 ms), and matching is also efficient (~2.11 ms per comparison). Off-line training for the metric and bone fragmentation takes longer (4 seconds for metric, 195 minutes for bone fragmentation).

Limitations and Future Work

  • Training Time: The off-line training for bone fragmentation is computationally intensive (195 minutes).
  • Dependency on Skeleton Quality: The performance is inherently tied to the accuracy of the skeleton extraction from the RGB-D sensor. Noise in joint positions can still impact results.
  • Future Work: The paper suggests that the learned metric can also be used for attribute-based descriptions, which could be useful in certain applications. Further exploration of how to optimize the bone fragmentation process and potentially reduce its training time would be beneficial.

Summary updated

BibTeX
@inproceedings{baltieri2013learning,
  author = {Davide Baltieri and Roberto Vezzani and Rita Cucchiara},
  title = {{Learning articulated body models for people re-identification}},
  booktitle = {ACM Multimedia},
  year = {2013},
  doi = {10.1145/2502081.2502147},
}
Figure