Abstract
Smartphone sensors based human activity recognition is attracting increasing interests nowadays with the popularization of smartphones. With the high sampling rates of smartphone sensors, it is a highly long-range temporal recognition problem, especially with the large intra-class distances such as the smartphones carried at different locations such as in the bag or on the body, and the small inter-class distances such as taking train or subway. To address this problem, we propose a new approach, an Independently Recurrent Neural Network (IndRNN) based long-term temporal activity recognition with spatial and frequency domain features. Considering the periodic characteristics of the sensor data, short-term temporal features are first extracted in the spatial and frequency domains. Then the IndRNN, which is able to capture long-term patterns, is used to further obtain the long-term features for classification. In view of the large differences when the smartphone is carried at different locations, a group based location recognition is first developed to pinpoint the location of the smartphone. The Sussex-Huawei Locomotion (SHL) dataset from the SHL Challenge is used for evaluation. An earlier version of the proposed method has won the second place award in the SHL Challenge 2020 (the first place if not considering multiple models fusion approach). The proposed method is further improved in this paper and achieves 80.72% accuracy, better than the existing methods using a single model.
Introduction
Human Activity Recognition (HAR) using smartphones faces significant hurdles due to the high sampling rates of modern sensors (typically 100Hz). This leads to extremely long sequences where traditional Recurrent Neural Networks (RNNs) often fail due to vanishing or exploding gradients. Furthermore, the system must handle high intra-class variance—where the same activity looks different depending on phone placement (e.g., bag vs. pocket)—and low inter-class variance, such as the subtle vibrational differences between a train and a subway.
This paper introduces a robust framework that leverages Independently Recurrent Neural Networks (IndRNN) combined with dual-domain feature extraction to achieve location-independent and user-independent recognition.
Proposed Framework
The proposed method follows a multi-stage pipeline designed to standardize raw sensor data and extract meaningful patterns across different time scales.
1. Coordinate Derotation
Raw sensor data is initially relative to the phone's orientation. To ensure consistency, the system uses quaternions to transform accelerometer and magnetometer data into a stable "World Coordinate System" (North-East-Down).
2. Dual-Domain Feature Extraction
Rather than processing raw time-series data, the framework segments signals into windows to extract features in two domains:
- Spatial Domain: Statistical measures such as mean, standard deviation, and range are calculated to capture the general intensity of movement.
- Frequency Domain: Fast Fourier Transform (FFT) amplitude spectrums are generated. This is crucial because human activities like walking or running are inherently periodic, and different transportation modes have unique mechanical resonance signatures.
3. Deep Dense IndRNN
The core architecture utilizes the IndRNN, where neurons in a layer are independent of each other. This independence allows the network to be significantly deeper and more stable when processing long-range temporal dependencies. The authors employ a "Dense" architecture, where each layer receives inputs from all preceding layers, facilitating better feature reuse and gradient flow.
4. Location-Aware Transfer Learning
Recognizing that phone placement significantly impacts sensor readings, the system first identifies the general location group of the device (e.g., "on-body" vs. "hand/bag"). It then employs transfer learning to fine-tune the activity recognition model based on this contextual information.
Experimental Results
The model was evaluated on the Sussex-Huawei Locomotion (SHL) dataset, which contains seven months of data across eight activity classes and four body positions.
Key findings include:
- State-of-the-Art Accuracy: The "Dense-IndRNN-aug" model achieved 80.72% accuracy, the highest reported for a single-model approach on this benchmark.
- Superiority over Baselines: The method significantly outperformed standard architectures, such as CNN+LSTM combinations (52.8%) and 3-Layer CNNs (76.4%).
- Ablation Study: The inclusion of "Dense" connections improved performance from 69.48% to 71.80%, while temporal feature augmentation further boosted it to 74.06% prior to the final transfer learning stage.
Conclusion
By combining the stability of IndRNNs with the descriptive power of frequency-domain features, this framework effectively addresses the long-term temporal challenges of HAR. While the model currently faces some confusion between similar motorized transport modes (e.g., Bus vs. Car), its performance as a single-model solution sets a new benchmark for the field. Future work will focus on refining spatial processing to better distinguish between these similar transportation classes.
Summary updated