TL;DR
This paper introduces HDNet, a novel hierarchical network for gait recognition using millimeter-wave (mmWave) radar. HDNet leverages a new "point flow" descriptor to capture dynamic inter-frame information and a dynamic frame sampling module to improve efficiency, achieving state-of-the-art results on mmWave gait recognition benchmarks.
What Problem Does This Paper Address?
Gait recognition, the process of identifying individuals by their walking style, has many applications, from personalized services to surveillance. While RGB cameras are common for this task, they struggle in poor lighting and raise privacy concerns. Millimeter-wave (mmWave) radar offers a promising alternative due to its robustness, non-invasiveness, and privacy-preserving nature. However, existing mmWave gait recognition methods primarily focus on aggregating spatial-temporal features within frames, neglecting the crucial dynamic information that exists between frames, which is essential for understanding the periodic nature of human walking. This paper aims to bridge this gap by developing a method that effectively captures and utilizes these inter-frame dynamics.
Methods
The proposed HDNet architecture comprises three main components:
- Spatial GNN Backbone: This module uses a graph neural network (specifically AdaptConv) to extract spatial features from individual radar point cloud frames. It's designed to capture relationships between points from different parts of the human body.
- Dynamic Frame Sampling (DFS): To improve computational efficiency and focus on informative frames, HDNet introduces a dynamic frame sampling module. This module learns to predict which frames are most relevant for gait recognition and prunes less informative ones. It uses a Gumbel-Softmax reparameterization technique to enable end-to-end training of this sampling process. A "mask loss" is also introduced to constrain the ratio of sampled frames.
- Temporal Aggregator (TA): This component utilizes a Transformer-based architecture to aggregate the extracted spatial features and the novel "point flow" features across the sampled frames. Transformers are chosen for their ability to model long-range dependencies, which is well-suited for the periodic nature of gait.
A key novelty is the point flow descriptor. This descriptor captures the dynamic changes between consecutive radar point cloud frames by calculating the difference in Doppler velocity between corresponding points in adjacent frames. This provides a new dimension of information about the walking motion.
Datasets
The authors evaluate HDNet on two public mmWave radar-based gait recognition datasets:
- mmGait [3]: A large dataset containing 30 hours of 3D point cloud data from 95 volunteers, collected using TI IWR1443 and TI IWR6843 mmWave sensing devices. It includes both fixed and free-route walking trajectories.
- STPointGCN [5]: A dataset with 75 minutes of 3D point cloud data from 6 participants, collected in three different environments using TI IWR6843.
For experiments, the authors focus on scenarios with a single person walking to simplify the problem and ensure fair comparison with other methods.
Key Insights and Ideas
- Point Flow for Dynamics: The introduction of "point flow" as a feature descriptor is a significant contribution. It effectively captures the subtle, dynamic changes between frames that are crucial for gait recognition, moving beyond static spatial-temporal feature aggregation.
- Dynamic Frame Sampling for Efficiency: The DFS module is a clever way to reduce computational load without sacrificing accuracy. By adaptively selecting informative frames, it makes the model more efficient, especially for longer sequences.
- Transformer's Suitability for Gait: The success of the Transformer-based Temporal Aggregator highlights its power in modeling the periodic and long-range dependencies inherent in human gait.
- mmWave Radar's Potential: The paper further solidifies mmWave radar as a viable and powerful sensor for gait recognition, demonstrating its ability to achieve high accuracy in a privacy-preserving manner.
Key Results
HDNet demonstrates superior performance on both datasets:
- On the mmGait dataset, HDNet achieves an accuracy of 96.52%, outperforming the previous state-of-the-art model SRPNet by 13.7% in accuracy.
- On the STPointGCN dataset, HDNet achieves an accuracy of 91.40%, outperforming the SOTA model STPointGCN by 21% in accuracy.
- The Dynamic Frame Sampling module shows significant benefits. Even when the sampling ratio is reduced to 0.3, HDNet with DFS maintains an accuracy above 90%, significantly outperforming random sampling at the same ratio.
Limitations and Future Work
- Single-Person Focus: The current evaluation primarily uses data with only one person walking. While this simplifies comparison, real-world scenarios often involve multiple people. Future work could explore extending HDNet to handle multi-target scenarios more robustly.
- Dataset Size: While the datasets are substantial, the authors acknowledge the need for more diverse and larger datasets to further validate the model's generalization capabilities.
- Computational Cost: Although DFS improves efficiency, the overall computational cost of HDNet might still be a consideration for real-time, resource-constrained applications. Further optimization could be explored.
Summary updated