WiTransformer: A Novel Robust Gesture Recognition Sensing Model with WiFi

Mingze Yang, Hai Zhu, Runzhe Zhu, Fei Wu, Ling Yin, Yuncheng YangSensors 2023 · DOI

TL;DR

This paper introduces WiTransformer, a novel deep learning model for human gesture recognition using WiFi signals. By leveraging a pure Transformer architecture and a domain-independent feature called Body-coordinate Velocity Profile (BVP), WiTransformer achieves robust performance across varying task complexities, outperforming existing methods.

What Problem Does This Paper Address?

Human Activity Recognition (HAR) using WiFi signals has emerged as a promising, non-invasive sensing technology. However, existing methods often struggle with increasing task complexity, such as a larger number of gestures, similar-looking actions, and signal distortions. This complexity diminishes the performance of HAR systems. Previous approaches have largely focused on improving accuracy through sophisticated models but have neglected the robustness needed for real-world scenarios with varying task demands. Furthermore, collecting large-scale, diverse datasets for WiFi-based HAR is challenging, making it difficult to train models that generalize well across different environments and users.

Methods

The core innovation of WiTransformer lies in its use of a pure Transformer architecture, eschewing traditional convolutional and recurrent neural networks. This approach is inspired by the success of Transformers in other domains like NLP and computer vision. To address the data-hungry nature of Transformers, the authors utilize the Body-coordinate Velocity Profile (BVP), a domain-independent feature derived from Channel State Information (CSI). BVP is designed to be more robust to environmental variations.

Two Transformer-based architectures are proposed:

  • Separated Spatiotemporal Model (SST): This model uses two separate Transformer encoders to independently extract spatial and temporal features from the BVP sequences.
  • United Spatiotemporal Model (UST): This model employs a single, unified spatiotemporal Transformer encoder. It utilizes novel techniques like "stacking-fusion" and "tube embedding" to fuse spatial and temporal information early on, allowing for more efficient processing and a single-dimensional encoder. UST also introduces "Channel Position Embedding (CPE)" to compensate for the lack of inherent order in BVP data.

Datasets

The experiments were conducted on the Widar 3.0 public dataset. This dataset comprises 22 gestures from 17 users across 8 locations and 5 orientations in 3 rooms. To evaluate robustness to task complexity, the original dataset was reconstructed into four Task Datasets (TDSs):

  • TDSs-6: 6 non-confusing gestures.
  • TDSs-9: 9 gestures with some similar gestures.
  • TDSs-12: 12 gestures with similar gestures and distortion.
  • TDSs-22: 22 gestures encompassing all task complexities.

Key Insights and Ideas

  • Transformers for WiFi HAR: The paper demonstrates that pure Transformer architectures, without traditional CNN or RNN backbones, can be highly effective for WiFi-based gesture recognition.
  • BVP as a Domain-Independent Feature: The Body-coordinate Velocity Profile (BVP) proves crucial for reducing the reliance on large-scale, cross-domain datasets, enabling better generalization.
  • UST's Unified Approach: The United Spatiotemporal Transformer (UST) architecture, with its early fusion of spatiotemporal information, offers superior performance and robustness compared to the separated approach (SST) and other backbone models.
  • Robustness to Task Complexity: UST exhibits remarkable robustness, with only a minor drop in accuracy as task complexity increases, a significant advantage over other methods.
  • SST's Limitations: The separated approach (SST) struggles due to the inherent lack of inductive bias in Transformers and the limited scale of the training data, leading to poor performance.

Key Results

  • Accuracy: UST achieved a recognition accuracy of 86.16% on the most complex task dataset (TDSs-22).
  • Robustness: When task complexity increased from TDSs-6 to TDSs-22, UST's accuracy decreased by only 3.18%, significantly outperforming other models which saw decreases of 7-21%.
  • Comparison: UST outperformed popular backbone models like CNN+LSTM, CNN+GRU, Conv2D, and Conv3D on more complex task datasets.
  • Efficiency: UST demonstrated lower computational complexity and inference time compared to SST and Conv3D, making it more efficient.

Limitations and Future Work

  • SST Failure: The paper notes that SST fails due to a lack of inductive bias and limited training data, a prediction that was largely confirmed by experiments.
  • Data Imbalance: While the experiment was not significantly affected unless the sample ratio was extreme (1:10), data imbalance was acknowledged as a potential factor influencing SST's performance.
  • Future Work: The authors aim to design better cross-domain cognition methods independent of BVPs and to develop more credible and efficient large-scale datasets. They also suggest that the Transformer architecture could be used for multi-modality fusion (e.g., combining WiFi with natural language or images) and applied to other dynamic recognition tasks like video analysis and remote sensing.

Summary updated

BibTeX
@article{yang2023witransformer,
  author = {Mingze Yang and Hai Zhu and Runzhe Zhu and Fei Wu and Ling Yin and Yuncheng Yang},
  title = {{WiTransformer: A Novel Robust Gesture Recognition Sensing Model with WiFi}},
  journal = {Sensors},
  year = {2023},
  doi = {10.3390/s23052612},
}
Figure