CarGait: Cross-Attention based Re-ranking for Gait recognition

Gavriel Habib, Noa Barzilay, Or Shimshi, Rami Ben-Ari, Nir Darshan

Introduction

Gait recognition—identifying individuals by their unique walking patterns—is a critical retrieval task in surveillance and forensic analysis. Most state-of-the-art models currently employ a "single-stage" approach, compressing entire walking sequences into global feature vectors. While efficient for large-scale database searches, these global representations often lack the granularity required to distinguish between "hard negatives"—different individuals with similar body shapes or stride dynamics. Consequently, while the correct identity may appear in the top retrieval results, it frequently fails to reach the Rank-1 position.

CarGait addresses this limitation by introducing a lightweight, second-stage re-ranking method. By analyzing fine-grained interactions between specific body parts using cross-attention, CarGait refines initial retrieval results, significantly boosting accuracy and surpassing existing re-ranking techniques in both precision and computational efficiency.

Methods

CarGait is designed as a modular re-ranking component that can be integrated with various pre-trained gait recognition models. The architecture focuses on "gait strips"—horizontal segments of a walking silhouette representing different body parts (e.g., head, torso, legs).

The CarGait architecture featuring the training and inference schemes, highlighting the cross-attention mechanism and dual-loss objectives.

The process involves several key steps:

  1. Feature Extraction: Initial feature maps are extracted from a frozen pre-trained gait model ($M$). These maps are organized into spatial strips ($F_p$ for probe, $F_c$ for candidate).
  2. Cross-Attention Mechanism: A transformer-based cross-attention module treats the probe strips as Queries ($Q$) and the candidate strips as Keys ($K$) and Values ($V$). This allows the model to learn specific correlations between body parts across different sequences.
  3. Dual-Stage Training: The re-ranker is optimized using two primary loss functions:
  4. Ranking Loss: A metric-learning objective that ensures the correct match is closer to the probe than incorrect candidates in the refined feature space.
  5. Classification Loss (CE loss): Acts as a regularizer to maintain identity-specific information.
  6. Inference: During deployment, a standard global search first retrieves the top-$K$ (typically $K=10$) candidates. CarGait then performs pairwise re-scoring only on these candidates, providing a final, more accurate ranking.

Datasets and Evaluation

The model was rigorously evaluated on three major gait benchmarks:

  • Gait3D: A challenging "in-the-wild" dataset captured from 39 cameras in a supermarket environment.
  • GREW: A massive-scale outdoor dataset containing over 128,000 sequences in unconstrained settings.
  • OU-MVLP: A large-scale indoor dataset featuring multiple viewing angles in a controlled environment.

Key Results

CarGait consistently improves Rank-1 accuracy across seven different base architectures, including CNN-based (e.g., GaitPart) and Transformer-based (e.g., SwinGait3D) models.

Radar charts illustrating the consistent performance gains (Rank-1 accuracy) achieved by CarGait across multiple datasets and base models.

Key performance highlights include:

  • Significant Accuracy Gains: On the GREW dataset, applying CarGait to the GaitBase model improved Rank-1 accuracy from 60.1% to 67.2%.
  • Efficiency: CarGait is remarkably fast, requiring only 6.52 ms per probe for a top-10 list, significantly outperforming traditional re-rankers like k-reciprocal encoding (214 ms) or GCR (1866 ms).
  • Verification Performance: In verification tasks, CarGait showed substantial improvements in True Positive Rates (TPR) at low False Positive Rates (FPR). For instance, GaitSet on Gait3D saw a jump from 27.1% to 39.3%.

Limitations and Future Work

While highly effective, CarGait requires a dedicated training phase for each base model it supports, unlike training-free methods. Additionally, its performance is bounded by the initial retrieval stage; if the correct identity is not within the top-$K$ candidates, the re-ranker cannot recover it. Future research may explore integrating temporal dynamics more deeply within the cross-attention blocks to further enhance discriminative power.

Summary updated

BibTeX
@inproceedings{habib2025cargait,
  author = {Gavriel Habib and Noa Barzilay and Or Shimshi and Rami Ben-Ari and Nir Darshan},
  title = {{CarGait: Cross-Attention based Re-ranking for Gait recognition}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2025},
  url = {https://openreview.net/forum?id=bbNAKbsCPR},
}
Figure