Walking Further: Semantic-aware Multimodal Gait Recognition Under Long-Range Conditions

Zhiyang Lu, Wen Jiang, Tianren Wu, Zhichao Wang, Changwang Zhang, Siqi Shen, Ming Cheng

Abstract

Gait recognition is an emerging biometric technology that enables non-intrusive and hard-to-spoof human identification. However, most existing methods are confined to short-range, unimodal settings and fail to generalize to long-range and cross-distance scenarios under real-world conditions. To address this gap, we present LRGait, the first LiDAR-Camera multimodal benchmark designed for robust long-range gait recognition across diverse outdoor distances and environments. We further propose EMGaitNet, an end-to-end framework tailored for long-range multimodal gait recognition. To bridge the modality gap between RGB images and point clouds, we introduce a semantic-guided fusion pipeline. A CLIP-based Semantic Mining (SeMi) module first extracts human body-part-aware semantic cues, which are then employed to align 2D and 3D features via a Semantic-Guided Alignment (SGA) module within a unified embedding space. A Symmetric Cross-Attention Fusion (SCAF) module hierarchically integrates visual contours and 3D geometric features, and a Spatio-Temporal (ST) module captures global gait dynamics. Extensive experiments on various gait datasets validate the effectiveness of our method.

The Challenge of Long-Range Gait Recognition

Traditional gait recognition systems are often "near-sighted," typically operating within 15 meters and relying solely on RGB video. In real-world surveillance, identifying individuals at distances up to 50 meters is crucial. However, long-range conditions introduce two significant data quality issues:

  1. RGB Degradation: As distance increases, images become blurred, and fine-grained details like facial features or clothing textures vanish.
  2. LiDAR Sparsity: LiDAR sensors produce fewer "hits" on a person at long range, resulting in extremely sparse point clouds that appear as disconnected dots.
Visualizing the degradation of LiDAR and Camera data as distance increases from 10m to 50m.

Furthermore, fusing these modalities is difficult because 2D pixels and 3D coordinates represent information differently. Existing methods often lose detail during preprocessing, such as when converting point clouds into 2D depth maps, leading to poor performance in unconstrained environments.

LRGait: A New Multimodal Benchmark

To facilitate research in this area, the authors introduced LRGait, a massive new benchmark featuring:

  • 101 subjects and 5,280 sequences.
  • Data captured at five distances (10m to 50m) from eight viewpoints.
  • Variations in lighting (day/night), weather, and carried objects.

Compared to existing datasets like SUSTech1K (limited to <12m) and FreeGait (max 25m), LRGait provides the first comprehensive testbed for true long-range scenarios.

EMGaitNet: Semantic-Guided Multimodal Fusion

The proposed EMGaitNet is an end-to-end framework designed to handle raw RGB and LiDAR data simultaneously. The core innovation is a semantic-guided pipeline that uses high-level concepts to bridge the modality gap.

The EMGaitNet architecture, featuring the SeMi, SGA, and SCAF modules for semantic-aware multimodal fusion.

The framework consists of several key components:

  • Semantic Mining (SeMi): Utilizing a CLIP-based module, the system extracts "body-part-aware" cues. By leveraging text prompts (e.g., "A photo of the [legs] of a person"), the model learns to identify specific regions like the head, arms, torso, and legs across both modalities.
  • Semantic-Guided Alignment (SGA): These body-part cues act as a universal translator. The SGA module uses cross-attention to align 2D visual features with 3D geometric features in a unified embedding space, ensuring spatial consistency between the video and the point cloud.
  • Symmetric Cross-Attention Fusion (SCAF): This module enables bidirectional communication between modalities. Camera data helps refine the sparse LiDAR structure, while LiDAR provides the geometric context missing in 2D images.
  • Spatio-Temporal (ST) Module: Finally, the system aggregates these features over time to capture the unique dynamics of an individual's gait.

Experimental Results

EMGaitNet demonstrates superior performance across multiple benchmarks:

  • LRGait: In the most challenging 50m scenario, it achieved 74.4% Rank-1 accuracy, a significant 14.0% improvement over the previous state-of-the-art.
  • SUSTech1K: Achieved 96.0% Rank-1 accuracy, outperforming existing methods by 2.2%.
  • Ablation Studies: Removing the SGA module caused accuracy to drop from 68.9% to 52.3%, highlighting that semantic guidance is the critical component for overcoming long-range data sparsity.

Conclusion and Future Directions

EMGaitNet proves that gait recognition at 50 meters is achievable by using high-level semantics to align disparate data sources. While the system excels in daylight, nighttime conditions remain a challenge due to the extreme domain shift in RGB data. Future research will likely explore multimodal domain adaptation to improve robustness in low-light environments and further scale the system for unconstrained real-world deployment.

Summary updated

Abstract

Gait recognition is an emerging biometric technology that enables non-intrusive and hard-to-spoof human identification. However, most existing methods are confined to short-range, unimodal settings and fail to generalize to long-range and cross-distance scenarios under real-world conditions. To address this gap, we present \textbf{LRGait}, the first LiDAR-Camera multimodal benchmark designed for robust long-range gait recognition across diverse outdoor distances and environments. We further propose \textbf{EMGaitNet}, an end-to-end framework tailored for long-range multimodal gait recognition. To bridge the modality gap between RGB images and point clouds, we introduce a semantic-guided fusion pipeline. A CLIP-based Semantic Mining (SeMi) module first extracts human body-part-aware semantic cues, which are then employed to align 2D and 3D features via a Semantic-Guided Alignment (SGA) module within a unified embedding space. A Symmetric Cross-Attention Fusion (SCAF) module hierarchically integrates visual contours and 3D geometric features, and a Spatio-Temporal (ST) module captures global gait dynamics. Extensive experiments on various gait datasets validate the effectiveness of our method.
BibTeX
@inproceedings{lu2026walking,
  author = {Zhiyang Lu and Wen Jiang and Tianren Wu and Zhichao Wang and Changwang Zhang and Siqi Shen and Ming Cheng},
  title = {{Walking Further: Semantic-aware Multimodal Gait Recognition Under Long-Range Conditions}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2026},
  doi = {10.1609/aaai.v40i9.37703},
}
Figure