Unsupervised Domain Adaptation for RF-Based Gesture Recognition

Bin-Bin Zhang, Dongheng Zhang, Yadong Li, Yang Hu, Yan Chen

TL;DR

This paper introduces a novel unsupervised domain adaptation framework for RF-based gesture recognition. By effectively leveraging unlabeled data from new environments, the proposed method significantly improves gesture recognition accuracy, overcoming performance degradation issues common in real-world deployments.

What Problem Does This Paper Address?

Gesture recognition using Radio Frequency (RF) signals (like WiFi or mmWave) offers advantages over traditional methods like cameras, including privacy and broader coverage. However, a major hurdle is that models trained in one specific environment (the "source domain") perform poorly when deployed in a new, different environment (the "target domain"). This performance drop is due to discrepancies in RF signals caused by varying collection conditions, such as the environment, subject, location, or orientation. Existing solutions often rely on supervised learning, which is impractical for target domains where labeled data is scarce. This paper tackles the challenge of adapting RF gesture recognition models to new, unseen conditions using only unlabeled data from those target domains.

Methods

The core of the proposed solution is an unsupervised domain adaptation (UDA) framework. The authors introduce several key components:

  • Pseudo-Labeling: This technique generates "pseudo labels" for the unlabeled target domain data. The model, trained on labeled source data, makes predictions on unlabeled target data. Predictions with high confidence are treated as ground truth labels, allowing the model to learn from this augmented dataset.
  • Confidence Constraint Loss: To mitigate the negative impact of potentially incorrect pseudo labels, a confidence constraint loss is introduced. This loss encourages smoother output probabilities, preventing the model from becoming overly confident in potentially wrong predictions.
  • Consistency Regularization: This method ensures that the model's predictions remain consistent even when the input data is slightly perturbed. To adapt this to RF signals, where semantic information is not easily interpretable like in images, the authors propose two novel data augmentation techniques:
  • Local Feature Erasing: Randomly removes parts of informative local features identified by a CFAR algorithm.
  • Time Erasing: Randomly removes frames from the time-series data, simulating gestures at different velocities.
  • Cross-Match Loss: This novel loss function elegantly integrates pseudo-labeling and consistency regularization. Instead of optimizing them separately, it directly computes the discrepancy between the pseudo labels and the predictions from augmented target data. This simplifies the framework and reduces computational burden.

The overall framework combines a CNN-GRU architecture for gesture recognition with these UDA techniques, optimizing a combined loss function.

Datasets

The proposed framework was evaluated on two public RF gesture recognition datasets:

  • Widar 3.0 Dataset: A WiFi-based dataset featuring 6 gesture categories collected across 3 different environments, 5 locations, and 5 orientations.
  • mmWave Radar Dataset: A mmWave-based dataset with 6 gesture categories collected from 25 volunteers across 6 environments and 5 locations.

Key Insights and Ideas

  • Leveraging Unlabeled Data is Crucial: The paper strongly emphasizes that the performance bottleneck in real-world RF gesture recognition stems from the inability to effectively use unlabeled data in target domains. Their UDA approach directly addresses this.
  • RF Data Augmentation is Non-Trivial: Unlike image data, direct application of standard augmentation techniques (like rotation) can corrupt the semantic meaning of RF signals. The proposed "Local Feature Erasing" and "Time Erasing" are tailored to the characteristics of RF data.
  • Confidence Matters for Pseudo-Labels: Simply generating pseudo-labels isn't enough. The confidence constraint loss is key to ensuring that the model learns from reliable pseudo-labels and avoids propagating errors.
  • Efficiency Through Integration: The "Cross-Match Loss" is a clever design choice that unifies two powerful UDA components, leading to a simpler and more computationally efficient framework without sacrificing performance.

Key Results

The proposed unsupervised domain adaptation framework achieved state-of-the-art results on both the WiFi and mmWave datasets across various cross-domain evaluation scenarios (orientation, location, subject, and environment).

  • WiFi Dataset: Achieved an average accuracy improvement of 4.35% compared to state-of-the-art methods.
  • mmWave Dataset: Achieved an average accuracy improvement of 2.25% compared to state-of-the-art methods.
  • The framework demonstrated significant accuracy gains across all tested domain shifts, highlighting its robustness and generalizability.
  • The cross-match loss reduced training time by approximately 15% compared to separate optimization of self-supervised and consistency losses.

Limitations and Future Work

  • Computational Cost: While the cross-match loss improves efficiency, the paper acknowledges that the UDA framework still incurs a slightly longer training time per epoch compared to the backbone model due to the addition of unlabeled data and augmentation.
  • Threshold Tuning: The effectiveness of pseudo-labeling is sensitive to the confidence threshold. The paper suggests that an appropriate threshold needs to be determined experimentally for optimal performance.
  • Data Augmentation Specificity: The proposed data augmentation methods are designed for UDA in RF signals and may not be directly applicable to other tasks or data types.

Future work could explore further optimizations for training efficiency and investigate the applicability of these techniques to other RF-based sensing tasks.

Summary updated

Abstract

Human gesture recognition with radio frequency (RF) signals has attained acclaim due to the omnipresence, privacy protection, and broad coverage nature of RF signals. These gesture recognition systems rely on neural networks trained with a large number of labeled data. However, the recognition model trained with data under certain conditions would suffer from significant performance degradation when applied in practical deployment, which limits the application of gesture recognition systems. In this article, we propose an unsupervised domain adaptation framework for RF-based gesture recognition aiming to enhance the performance of the recognition model in new conditions by making effective use of the unlabeled data from new conditions. We first propose pseudo labeling and consistency regularization to utilize unlabeled data for model training and eliminate the feature discrepancies in different domains. Then we propose a confidence constraint loss to enhance the effectiveness of pseudo labeling, and design two corresponding data augmentation methods based on the characteristic of the RF signals to strengthen the performance of the consistency regularization, which can make the framework more effective and robust. Furthermore, we propose a cross-match loss to integrate the pseudo labeling and consistency regularization, which makes the whole framework simple yet effective. Extensive experiments demonstrate that the proposed framework could achieve 4.35% and 2.25% accuracy improvement comparing with the state-of-the-art methods on public WiFi data set and millimeter wave (mmWave) radar data set, respectively.
BibTeX
@article{zhang2023unsupervised,
  author = {Bin-Bin Zhang and Dongheng Zhang and Yadong Li and Yang Hu and Yan Chen},
  title = {{Unsupervised Domain Adaptation for RF-Based Gesture Recognition}},
  journal = {IEEE Internet of Things Journal},
  year = {2023},
  url = {https://openreview.net/forum?id=Sv8wI5f7lG},
}
Figure