Abstract
Gait is a unique biometric feature that can be recognized at a distance, offering broad applications in crime prevention and forensic identification. Traditional gait recognition methods rely on either templates, which struggle to preserve temporal information, or sequences, which impose rigid constraints that reduce flexibility. This paper introduces GaitSet, a novel perspective that treats gait as a "deep set." By integrating a set of gait frames through a global-local fused deep network, the model becomes immune to frame permutations and can naturally integrate frames from diverse scenarios. Experiments demonstrate state-of-the-art performance, achieving 96.1% rank-1 accuracy on CASIA-B and 87.9% on OU-MVLP. Notably, GaitSet remains highly robust under complex conditions like coat-wearing and can identify individuals with as few as seven random frames.
Introduction
Gait recognition—identifying individuals by their walking style—is a powerful biometric because it works at a distance without requiring the subject's cooperation. However, traditional methods often struggle with changes in viewpoint, clothing, or carrying conditions.
Existing approaches generally fall into two categories: gait templates and gait sequences. Gait templates, like Gait Energy Images (GEI), compress a walking cycle into a single image. While computationally efficient, they lose vital temporal and fine-grained spatial information. Conversely, gait sequences treat gait as a video, using 3D convolutions or Recurrent Neural Networks (RNNs). While these preserve temporal data, they are often "brittle," requiring specific frame rates and continuous sequences.
The authors of GaitSet propose a paradigm shift: stop treating gait as a rigid video sequence and start treating it as an unordered set of independent silhouettes. The key insight is that while walking is periodic, the unique set of silhouettes a person produces is more discriminative than the specific order of their steps.
Methods: The GaitSet Architecture
The GaitSet architecture follows a three-step pipeline designed to extract and aggregate features from a "bag of silhouettes."
1. Frame-level Feature Extraction
A standard 2D Convolutional Neural Network (CNN) processes each silhouette in the input set independently. This stage extracts "local" spatial features from every frame before any temporal or set-level aggregation occurs.
2. Set Pooling (SP)
This is the core innovation of the model. A Set Pooling operation (such as max, mean, or a learned attention mechanism) aggregates all frame-level features into a single "set-level" feature map. Because these operations are commutative, the model is permutation invariant—scrambling the input frames does not change the output. This allows the model to handle non-consecutive frames and varying frame rates seamlessly.
3. Horizontal Pyramid Mapping (HPM)
To capture both global body shape and local limb movements, the set-level feature map is split into horizontal strips at multiple scales. Each strip is processed by an independent fully connected layer to create a final, highly discriminative feature vector. This ensures that the model pays attention to specific body parts (like the swing of an arm or the stride of a leg) while maintaining a global view of the subject.
Additionally, a Multilayer Global Pipeline (MGP) pulls features from different depths of the CNN, ensuring the model captures both fine-grained details from shallow layers and abstract patterns from deeper layers.
Experimental Results
The model was evaluated on the two most prominent gait recognition benchmarks: CASIA-B (124 subjects, 11 angles, 3 conditions) and OU-MVLP (over 10,000 subjects).
- State-of-the-Art Accuracy: On CASIA-B, GaitSet reached 96.1% rank-1 accuracy under normal conditions. In the challenging "Coat" condition, where body shape is obscured, it achieved 70.3%, outperforming previous best methods by over 15%.
- Large-Scale Performance: On the massive OU-MVLP dataset, it achieved an average accuracy of 87.9%.
- Efficiency with Sparse Data: The model is remarkably robust to low frame counts. It maintains 85.0% accuracy on CASIA-B using only 7 frames per sequence, whereas traditional template-based methods often fail with such limited data.
Conclusion
GaitSet demonstrates that order invariance is a feature, not a bug, in gait recognition. By treating frames as a set, the model gains immense flexibility and robustness against missing data or environmental changes. While clothing remains a significant challenge for the field, the "Deep Set" perspective offers a promising path forward for video-based biometric tasks where temporal order may be less critical than the cumulative information within the set.
Summary updated