Abstract
Gait recognition is an emerging biometric technology that enables non-intrusive and hard-to-spoof human identification. However, most existing methods are confined to short-range, unimodal settings and fail to generalize to long-range and cross-distance scenarios under real-world conditions. To address this gap, we present LRGait, the first LiDAR-Camera multimodal benchmark designed for robust long-range gait recognition across diverse outdoor distances and environments. We further propose EMGaitNet, an end-to-end framework tailored for long-range multimodal gait recognition. To bridge the modality gap between RGB images and point clouds, we introduce a semantic-guided fusion pipeline. A CLIP-based Semantic Mining (SeMi) module first extracts human body-part-aware semantic cues, which are then employed to align 2D and 3D features via a Semantic-Guided Alignment (SGA) module within a unified embedding space. A Symmetric Cross-Attention Fusion (SCAF) module hierarchically integrates visual contours and 3D geometric features, and a Spatio-Temporal (ST) module captures global gait dynamics. Extensive experiments on various gait datasets validate the effectiveness of our method.
The Challenge of Long-Range Gait Recognition
Traditional gait recognition systems are often "near-sighted," typically operating within 15 meters and relying solely on RGB video. In real-world surveillance, identifying individuals at distances up to 50 meters is crucial. However, long-range conditions introduce two significant data quality issues:
- RGB Degradation: As distance increases, images become blurred, and fine-grained details like facial features or clothing textures vanish.
- LiDAR Sparsity: LiDAR sensors produce fewer "hits" on a person at long range, resulting in extremely sparse point clouds that appear as disconnected dots.
Furthermore, fusing these modalities is difficult because 2D pixels and 3D coordinates represent information differently. Existing methods often lose detail during preprocessing, such as when converting point clouds into 2D depth maps, leading to poor performance in unconstrained environments.
LRGait: A New Multimodal Benchmark
To facilitate research in this area, the authors introduced LRGait, a massive new benchmark featuring:
- 101 subjects and 5,280 sequences.
- Data captured at five distances (10m to 50m) from eight viewpoints.
- Variations in lighting (day/night), weather, and carried objects.
Compared to existing datasets like SUSTech1K (limited to <12m) and FreeGait (max 25m), LRGait provides the first comprehensive testbed for true long-range scenarios.
EMGaitNet: Semantic-Guided Multimodal Fusion
The proposed EMGaitNet is an end-to-end framework designed to handle raw RGB and LiDAR data simultaneously. The core innovation is a semantic-guided pipeline that uses high-level concepts to bridge the modality gap.
The framework consists of several key components:
- Semantic Mining (SeMi): Utilizing a CLIP-based module, the system extracts "body-part-aware" cues. By leveraging text prompts (e.g., "A photo of the [legs] of a person"), the model learns to identify specific regions like the head, arms, torso, and legs across both modalities.
- Semantic-Guided Alignment (SGA): These body-part cues act as a universal translator. The SGA module uses cross-attention to align 2D visual features with 3D geometric features in a unified embedding space, ensuring spatial consistency between the video and the point cloud.
- Symmetric Cross-Attention Fusion (SCAF): This module enables bidirectional communication between modalities. Camera data helps refine the sparse LiDAR structure, while LiDAR provides the geometric context missing in 2D images.
- Spatio-Temporal (ST) Module: Finally, the system aggregates these features over time to capture the unique dynamics of an individual's gait.
Experimental Results
EMGaitNet demonstrates superior performance across multiple benchmarks:
- LRGait: In the most challenging 50m scenario, it achieved 74.4% Rank-1 accuracy, a significant 14.0% improvement over the previous state-of-the-art.
- SUSTech1K: Achieved 96.0% Rank-1 accuracy, outperforming existing methods by 2.2%.
- Ablation Studies: Removing the SGA module caused accuracy to drop from 68.9% to 52.3%, highlighting that semantic guidance is the critical component for overcoming long-range data sparsity.
Conclusion and Future Directions
EMGaitNet proves that gait recognition at 50 meters is achievable by using high-level semantics to align disparate data sources. While the system excels in daylight, nighttime conditions remain a challenge due to the extreme domain shift in RGB data. Future research will likely explore multimodal domain adaptation to improve robustness in low-light environments and further scale the system for unconstrained real-world deployment.
Summary updated