TL;DR
This paper introduces a novel method for individual recognition using 3D skeletal locomotion data by representing human movement on a Riemannian manifold. This approach captures unique spatio-temporal features that are more discriminative than traditional methods, leading to state-of-the-art performance in gait recognition tasks.
What Problem Does This Paper Address?
Individual recognition based on locomotion, or gait, is a challenging computer vision problem with applications in security, human-computer interaction, and surveillance. While gait is a promising biometric due to its non-contact nature and inherent individuality, existing methods often struggle with variations in illumination, viewpoint, and appearance. Furthermore, many approaches fail to explicitly capture the subtle, intrinsic differences in how individuals move. This paper aims to address this by developing a feature representation that explicitly models these unique spatio-temporal characteristics of human locomotion.
Methods
The core of this work lies in representing human poses and their temporal evolution on a Riemannian manifold. This is a departure from traditional Euclidean space representations.
- Riemannian Manifold Representation: Human skeletons are modeled as points on a Riemannian manifold. This allows for a more natural representation of the complex, non-linear relationships between joint positions and orientations.
- Aligned Riemannian Mean Motion Sequence (ARMMS): The authors first compute a "mean motion sequence" by aligning related motion sequences on the Riemannian manifold. This ARMMS serves as a common reference.
- Spatial State Descriptors: Differences between an individual's motion sequence and the ARMMS are then modeled as spatial state descriptors. These capture how an individual's pose deviates from the average.
- Temporal Hierarchy of Covariance Descriptors: To capture temporal variations, a hierarchical structure of covariance matrices is imposed on the spatial state descriptors. This creates a higher-order statistical representation of the motion's dynamics over time.
- Kernel Metric Learning: Finally, a kernel metric learning approach, specifically a kernelized version of Large Margin Nearest Neighbor (LMNN), is employed. This maps the spatio-temporal features into a high-dimensional space to learn a low-rank distance metric, enhancing discriminative power and improving classification accuracy. The Radial Basis Function (RBF) kernel is used for this mapping.
Datasets
The proposed method was evaluated on three datasets:
- CMU Mocap database: A large dataset of motion capture data.
- UPCV Gait database: A dataset specifically designed for gait recognition, featuring walking sequences from 30 individuals captured with a Kinect sensor.
- Custom-built database: The authors also constructed their own database using a digital camera to capture walking and running sequences from 20 individuals. This dataset was used to analyze the impact of external factors like shoes and carrying conditions on gait recognition.
Key Insights and Ideas
- Riemannian Geometry for Gait: The fundamental insight is that human locomotion has an intrinsic geometric structure that is better captured by Riemannian geometry than by Euclidean space. This allows for a more nuanced representation of subtle individual differences.
- Mean Motion as a Reference: Using an "aligned Riemannian mean motion sequence" as a reference point is crucial. It provides a stable baseline against which individual deviations can be measured, effectively isolating identity-specific motion patterns.
- Hierarchical Covariance for Temporal Dynamics: Moving beyond simple frame-to-frame analysis, the hierarchical covariance descriptor captures how the spatial relationships evolve over time within specific segments of the motion, providing a richer temporal signature.
- Fusion of Spatial and Temporal Features: The success of the method hinges on effectively combining both the static spatial pose information (relative to the mean) and the dynamic temporal evolution of these poses.
Key Results
The proposed Riemannian spatio-temporal feature representation achieved state-of-the-art results across the evaluated datasets:
- CMU Mocap database: Achieved a 98.70% classification rate, outperforming previous methods by over 2%.
- UPCV Gait database: Achieved a 97.90% classification rate, exceeding the second-best approach by more than 1%.
- Custom database: Demonstrated robustness to various factors like different shoe types and carrying conditions, with recognition rates reaching up to 99.7% for walking sequences. The fusion of spatial and temporal features (KRGF + KRTHC) consistently outperformed individual feature sets.
Limitations and Future Work
- Computational Cost: While not explicitly detailed as a limitation, the use of Riemannian geometry and kernel methods can be computationally intensive.
- Dependence on Pose Estimation: For applications where 3D joint locations are not directly provided (e.g., using only RGB cameras), the performance is dependent on the accuracy of the underlying 3D pose estimation method.
- Future Work: The authors suggest that their approach, combined with deep learning-based pose estimation, can be used for real-world individual recognition scenarios. Further exploration into optimizing the kernel metric learning and exploring other manifold structures could also be promising.
Summary updated