TL;DR
This paper introduces SE-ABLSTM-C, a novel Wi-Fi-based system for recognizing human activities performed by individuals not seen during training (cross-person activity recognition or CPAR). By combining snapshot ensemble learning with a center loss metric learning approach, SE-ABLSTM-C significantly improves generalization capabilities, achieving application-level accuracy for recognizing seven distinct activities.
What Problem Does This Paper Address?
Human Activity Recognition (HAR) is crucial for applications like elderly monitoring and intrusion detection. While Wi-Fi-based systems offer non-contact and privacy-preserving advantages, existing deep learning methods often struggle with cross-person activity recognition (CPAR). This means they perform well when recognizing activities of people they were trained on, but fail when encountering new individuals whose activity patterns might differ. This lack of generalization limits their real-world applicability, as retraining for every new user is impractical. The paper aims to bridge this gap by developing a method that can accurately recognize activities of unknown individuals without requiring prior training data from them.
Methods
The proposed method, SE-ABLSTM-C, builds upon the Attention-based Bi-directional Long Short-Term Memory (ABLSTM) network, a state-of-the-art DNN for HAR. The novelty lies in two key components:
- Snapshot Ensemble Learning: Instead of training a single model, this technique trains multiple "base-classifiers" within a single training process. It leverages a cyclic cosine annealing learning rate schedule to save model weights at different points (local minima) during training. This creates a diverse set of models that, when combined, enhance generalization.
- Metric Learning with Center Loss: To further improve the ability to distinguish between activities, especially for unseen individuals, the researchers incorporate a center loss function. This metric learning approach encourages features from the same activity class to be clustered closely together in the feature space, while pushing features from different classes further apart. This makes the learned representations more discriminative.
The SE-ABLSTM-C method combines these two techniques, using the hybrid loss function (softmax loss + center loss) to train the ABLSTM network within the snapshot ensemble framework.
Datasets
The study utilizes a newly collected CSI dataset for training and evaluation.
- Our Dataset: This dataset was collected using ESP32 development boards and comprises Channel State Information (CSI) data from five male subjects performing seven activities (waving, clapping, walking, lying down, sitting down, falling, and picking up). The dataset contains 700 data points in total, with each subject repeating 20 rounds of the 7 activities. This is described as a "medium-scale" dataset.
The paper also references existing CSI datasets for HAR in Table 1, but these were not used for training or evaluation in this specific study.
Key Insights and Ideas
- Generalization is Key for Real-World CPAR: The core challenge addressed is moving beyond recognizing activities of known individuals to recognizing activities of unknown individuals. This highlights the critical need for robust generalization in HAR systems.
- Ensemble Learning for Diversity: Snapshot ensemble learning is a clever way to generate multiple diverse models from a single training run, effectively exploring different regions of the hypothesis space and improving robustness.
- Metric Learning for Discriminative Features: The center loss is crucial for making the learned features more distinct. This is particularly important for CPAR, where subtle variations in activity patterns between individuals can be challenging to capture.
- Wi-Fi CSI as a Promising Modality: The paper reinforces the potential of Wi-Fi CSI for non-contact, privacy-preserving activity recognition, especially in indoor environments.
Key Results
The proposed SE-ABLSTM-C method demonstrated significant improvements in cross-person activity recognition (CPAR) across five different training/testing patterns.
- Task I (Seen Persons): As a baseline, conventional HAR methods were evaluated. ABLSTM achieved the highest average accuracy of 97.23% for recognizing activities of seen persons.
- Task II (Unknown Persons - CPAR):
- The SE-ABLSTM-C method markedly improved recognition accuracies compared to conventional HAR methods (CNN, LSTM, ABLSTM) and even its own components (SE-ABLSTM and ABLSTM-C).
- For example, in the Pattern BCDE-A, the proposed SE-ABLSTM-C achieved an average accuracy of 86.68%, outperforming ABLSTM (83.57%), LSTM (83.29%), CNN (83.99%), and LAGMAT (85.67%).
- Across all five CPAR patterns, SE-ABLSTM-C achieved the highest average accuracies, ranging from 78.86% to 89.03%.
- The paper states that SE-ABLSTM-C improved the average accuracy of seven activities by approximately 3–14% compared to the ABLSTM method.
Limitations and Future Work
- Falling Activity Recognition: The paper acknowledges that the performance for recognizing the "falling" activity remains relatively unsatisfied, even with the proposed method. This is attributed to the similarity between falling and picking up, making them difficult to distinguish.
- Dataset Scale: The authors note that their dataset is of medium scale, and increasing the number of subjects could further enhance the study.
- Activity Similarity: The vulnerability of Wi-Fi-based recognition to similar activities is a general limitation.
- Future Work:
- Investigating CPAR for activities with similar categories.
- Addressing open-set problems where activities unknown during training might appear.
- Exploring parallel computing with Transformer modules for efficiency.
- Integrating Wi-Fi sensing with camera data for human gait trajectory generation.
- The authors suggest that in scenarios requiring robust recognition of certain activities, similar categories like falling and picking up might need to be excluded.
Summary updated