Abstract
Gait recognition is a biometric technique that identifies individuals by their unique walking styles, which is suitable for unconstrained environments and has a wide range of applications. While current methods focus on exploiting body part-based representations, they often neglect the hierarchical dependencies between local motion patterns. In this paper, we propose a hierarchical spatio-temporal representation learning (HSTL) framework for extracting gait features from coarse to fine. Our framework starts with a hierarchical clustering analysis to recover multi-level body structures from the whole body to local details. Next, an adaptive region-based motion extractor (ARME) is designed to learn region-independent motion features. The proposed HSTL then stacks multiple ARMEs in a top-down manner, with each ARME corresponding to a specific partition level of the hierarchy. An adaptive spatio-temporal pooling (ASTP) module is used to capture gait features at different levels of detail to perform hierarchical feature mapping. Finally, a frame-level temporal aggregation (FTA) module is employed to reduce redundant information in gait sequences through multi-scale temporal downsampling. Extensive experiments on CASIA-B, OUMVLP, GREW, and Gait3D datasets demonstrate that our method outperforms the state-of-the-art while maintaining a reasonable balance between model accuracy and complexity.
Introduction
Gait recognition—identifying individuals by the way they walk—is a powerful biometric because it works at a distance without requiring the subject's cooperation. However, it faces significant hurdles: changes in viewpoint, clothing (like heavy coats), and carrying objects (like bags) can drastically alter a person's silhouette.
Current state-of-the-art methods typically use "part-based" representations, slicing the human silhouette into horizontal strips. However, these methods often neglect the hierarchical dependency of motion. For example, the movement of a foot is physically constrained by the leg, which is part of the lower body. Existing models often treat these regions as independent or purely global, failing to capture the rich, multi-scale relationships between different body parts during a gait cycle.
Methodology
The HSTL framework introduces a "coarse-to-fine" approach to capture gait features through a multi-level hierarchy, moving from the whole body down to individual limbs.
Hierarchical Body Partitioning
Instead of arbitrary horizontal slicing, HSTL uses hierarchical clustering on motion distributions to identify semantically related regions. This process recovers multi-level body structures, defining the structural "levels" the model will process. By forcing the model to look at the "whole body," then "upper/lower body," and finally "limbs," it learns more robust features that are less sensitive to local occlusions (like a bag blocking part of a leg).
Core Components
- Adaptive Region-based Motion Extractor (ARME): These modules are stacked in a top-down manner. Each ARME uses 3D convolutions to learn motion patterns specific to a particular partition level, ensuring the model captures both global posture and fine-grained local mechanics.
- Adaptive Spatio-Temporal Pooling (ASTP): This module maps the features from different hierarchical levels into a unified gait embedding. It uses a non-uniform pooling strategy that better fits the varying characteristics of different body regions.
- Frame-level Temporal Aggregation (FTA): To handle the redundancy in video sequences, the FTA module uses multi-scale temporal downsampling and an attention-like weighting mechanism to compress the sequence into the most discriminative "key" features.
Experimental Evaluation
The authors evaluated HSTL on four major gait recognition benchmarks: CASIA-B, OUMVLP, GREW, and Gait3D. These datasets cover a range of challenges, from controlled viewpoint variations to "in-the-wild" surveillance scenarios.
Key Results
- Superior Accuracy: On CASIA-B, HSTL achieved a mean Rank-1 accuracy of 98.1% (Normal), 95.9% (Bag), and 88.9% (Coat), outperforming previous leaders like MetaGait and CSTL.
- Large-Scale Performance: On OUMVLP, it reached 92.4% accuracy, a significant 2.4% improvement over the previous best-performing model.
- Robustness "In the Wild": On the challenging Gait3D dataset, HSTL improved the Rank-1 accuracy from 48.7% to 61.3%, a massive leap for unconstrained gait recognition.
- Complexity Balance: The model maintains a lower FLOP (Floating Point Operations) count than many competing 3D-convolution-based methods while delivering higher accuracy, making it more viable for real-world deployment.
Conclusion
The HSTL framework advances gait recognition by modeling the human body as a nested system rather than a collection of independent strips. This hierarchical approach allows the model to achieve high accuracy without needing excessively deep or wide 3D convolutional networks. While the current pre-defined hierarchy is effective, future work may explore end-to-end learning of optimal body partitions and integration with 3D mesh-based data to further improve viewpoint invariance.
Summary updated