TL;DR
This paper introduces GaitSG, a novel gait recognition method that leverages the 3D human body model, SMPL, represented as a graph. By treating SMPL parameters as graph nodes and employing graph convolutional networks (GCNs) and a new part graph pooling block (PGPB), GaitSG effectively captures human pose and shape for more robust gait identification, outperforming existing methods in accuracy and convergence speed.
What Problem Does This Paper Address?
Gait recognition, the task of identifying individuals by their walking patterns, is crucial for security and surveillance. Traditional methods often rely on 2D silhouettes, which are sensitive to variations in clothing and viewpoint, limiting their effectiveness in real-world scenarios. Model-based methods, using skeletons or 3D human models like SMPL, offer better robustness. However, existing SMPL-based methods often treat SMPL parameters as a whole or use simple MLPs, failing to fully exploit the rich structural and shape information embedded within SMPL representations. This paper aims to unlock the full potential of SMPL models for gait recognition by representing them in a graph structure and developing specialized network components to extract discriminative features.
Methods
The proposed method, GaitSG, takes SMPL parameters as input and models them as graphs. The core components are:
- SMPL Feature Extractor Block (SFEB): This block processes SMPL parameters. Shape parameters are fed into an MLP, while pose parameters are treated as a graph and processed by a GCN. This allows for spatial feature extraction from both pose and shape.
- Part Graph Pooling Block (PGPB): This novel component addresses limitations of standard graph structures, such as the physical distance-unaware nature and difficulty in capturing long-range dependencies. PGPB explicitly incorporates human body priors by splitting the pose graph into different parts (horizontal, limb, and trunk) at multiple scales. This allows for a more discriminative representation by focusing on key body parts and their relationships.
- Graph Representation: SMPL pose parameters are represented as a graph where joints are nodes and connections are edges. Additional edges are added between the neck and collarbone nodes to better simulate real-life scenarios.
- Overall Pipeline: SFEB extracts spatial features from each frame. These features are then aggregated temporally using a pooling neck, which includes both a standard set pooling block and the proposed PGPB. Finally, an identification head maps the combined features into a discriminative space for gait identification.
Datasets
The paper evaluates GaitSG on two datasets:
- Gait3D: A large-scale, challenging outdoor dataset providing accurate 3D SMPL gait representations. It features real-world conditions like viewpoint variance and occlusion.
- CASIA-B: A popular in-the-lab dataset that provides RGB videos. SMPL models are generated from these videos using the ROMP method.
Key Insights and Ideas
- SMPL as a Graph: Representing SMPL pose parameters as a graph is a key innovation. This allows for the application of powerful graph convolutional networks, which are well-suited for capturing the relational structure of human joints.
- Explicit Viewpoint Encoding: The PGPB's part-splitting mechanism, particularly the trunk split, explicitly leverages human body priors to encode viewpoint information. This is crucial for improving robustness in varying camera angles.
- Holistic SMPL Utilization: By processing both shape and pose parameters separately and then combining them, GaitSG ensures that the model benefits from the complete information provided by SMPL, rather than relying on a single modality or a simplified representation.
- Faster Convergence: The graph-based approach and specialized blocks lead to significantly faster convergence during training compared to baseline methods, indicating a more efficient learning process.
Key Results
- Gait3D Performance: GaitSG achieves competitive results, outperforming the baseline SMPLGait (3D only) by approximately twice the Rank-1 accuracy and requiring three times fewer training iterations. GaitSG+ (using ST-GCNs) further improves performance.
- CASIA-B Performance: GaitSG demonstrates strong performance across different conditions (NM, BG, CL) and views, showing significant improvements over the baseline method.
- Ablation Studies:
- Using both pose and shape parameters in SFEB yields better results than using either alone, with pose parameters showing a stronger contribution.
- The PGPB's splits (horizontal, limb, trunk) progressively improve performance, highlighting the effectiveness of the part-based approach.
- The proposed viewpoint encoding method (trunk split) significantly outperforms methods without explicit view encoding or those relying on camera parameters.
Limitations and Future Work
- SMPL Generation Maturity: The authors acknowledge that the upstream SMPL generation technique is not yet fully mature, which can impact overall performance.
- Dataset Scarcity: There is a scarcity of gait datasets with SMPL modalities. The authors hope for more datasets to emerge to promote research in this area.
- Future Directions: Future work could involve incorporating advanced graph techniques, fusing SMPL with other gait modalities for even better identification, and exploring extended SMPL models (e.g., SMPL-X, SMPL-H) for more detailed human representations.
Summary updated