GaitCycFormer: Leveraging Gait Cycles and Transformers for Gait Emotion Recognition

Qingyang Zeng, Lin ShangAAAI 2025 · DOI

TL;DR

This paper introduces GaitCycFormer, a novel Graph Transformer architecture for recognizing human emotions from 3D skeleton gait data. It addresses the limitations of existing methods in capturing long-range temporal dependencies and robustness to varying sampling rates by introducing a gait cycle-aware position encoding and a bi-level Transformer.

What Problem Does This Paper Address?

Recognizing human emotions from gait (Gait Emotion Recognition or GER) is a challenging but promising area within affective computing. While 3D skeleton data offers a rich source of information, existing GER methods often struggle to effectively capture the complex spatial and temporal patterns inherent in human walking. Specifically, they tend to focus on local spatial features and fail to model long-distance temporal relationships within a gait sequence. Furthermore, many methods are sensitive to how gait sequences are sampled (e.g., frame rate), leading to significant performance drops when sequences are presented in a different order or at a different rate than during training. This lack of robustness limits their applicability in real-world scenarios.

Methods

The GaitCycFormer framework builds upon a Spatial-Temporal Graph Convolutional Network (ST-GCN) backbone to extract initial spatial-temporal features. The core innovation lies in its bi-level Transformer module, which is designed to capture both local and global temporal information more effectively.

Key components include:

  • Cycle Position Encoding (CPE): Instead of standard positional encodings, GaitCycFormer uses CPE, which is derived from accurately identified gait cycles. This encoding explicitly segments gait sequences into periodic units, allowing the model to better understand the cyclical nature of walking and model fine-grained temporal features within each cycle. A novel window-based method is proposed for calculating gait cycles, which dynamically adjusts its window size based on the sequence length to ensure more accurate cycle detection, even when sequences are downsampled.
  • Bi-level Transformer: This module consists of:
  • Intra-cycle Transformer: Focuses on capturing local temporal dependencies within a single gait cycle.
  • Inter-cycle Transformer: Captures global temporal information and relationships between different gait cycles. This hierarchical approach helps disentangle local movements from overall gait patterns.
  • Graph Transformer (GTT) Blocks: These blocks integrate GCN layers with the bi-level Transformer, allowing for the simultaneous extraction of spatial and temporal features. This hybrid approach leverages the strengths of both graph-based methods for spatial topology and Transformers for temporal modeling.

Datasets

The paper evaluates GaitCycFormer on two public datasets:

  • EGait: Contains 1,835 gait samples across four emotions (happy, sad, angry, neutral), with sequences of 240 frames.
  • EWalk: Includes 342 gait samples from 90 participants, with varying sequence lengths that are standardized to 240 frames by looping.

Key Insights and Ideas

  • Gait is inherently cyclic: Recognizing this periodicity is crucial for effective emotion recognition. Existing methods often overlook this, leading to suboptimal performance.
  • Decomposing temporal information is key: Separating local (intra-cycle) and global (inter-cycle) temporal modeling allows for a more nuanced understanding of gait dynamics.
  • Robust gait cycle calculation matters: The accuracy of gait cycle detection directly impacts the effectiveness of cycle-aware encoding and subsequent temporal modeling. The proposed window-based method offers improved robustness to downsampling.
  • Transformer's attention mechanism is powerful, but needs guidance: While Transformers excel at capturing long-range dependencies, applying them directly to raw gait sequences can lead to "over-globalizing" issues. GaitCycFormer's structured approach (bi-level Transformer, CPE) guides the attention mechanism effectively.

Key Results

GaitCycFormer achieves state-of-the-art performance on both the EGait and EWalk datasets.

  • On EGait, it achieves an accuracy of 86.3%, outperforming previous methods like STA-GCN (85.8%) and MSA-GCN (83.5%).
  • On EWalk, it achieves 83.4%, surpassing STA-GCN (82.8%).
  • The model demonstrates significant robustness to sampled sequence variations. In random sampling settings, its accuracy decline is minimal (less than 0.1% for 48 frames), unlike TCNs which show a more pronounced drop. This highlights its improved generalization capacity.
  • Ablation studies confirm the effectiveness of the GTT Block and the proposed Cycle Position Encoding, with CPE leading to better performance than vanilla positional encoding.

Limitations and Future Work

  • Limitations: The paper notes that while self-supervised learning methods are emerging, they still lag behind fully supervised methods in performance. The current framework is primarily focused on supervised learning. The "over-globalizing problem" in Transformers is acknowledged, and while GaitCycFormer mitigates it, further exploration might be beneficial.
  • Future Work: The authors suggest that their framework can serve as a valuable reference for other skeleton-based gait tasks. Further research could explore more advanced self-supervised learning techniques within this framework or investigate its applicability to other human motion analysis tasks beyond emotion recognition.

Summary updated

BibTeX
@inproceedings{zeng2025gaitcycformer,
  author = {Qingyang Zeng and Lin Shang},
  title = {{GaitCycFormer: Leveraging Gait Cycles and Transformers for Gait Emotion Recognition}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i9.33064},
}
Figure