Abstract
Smartphone-based human activity recognition (HAR) faces significant challenges due to high sensor sampling rates, large intra-class distances (varying phone placements), and small inter-class distances (similar transportation modes). We propose a framework that combines short-term spatial and frequency feature extraction with a long-term Independently Recurrent Neural Network (IndRNN). By extracting periodic characteristics in both spatial and frequency domains and leveraging IndRNN's ability to capture long-term patterns, our method achieves 80.72% accuracy on the Sussex-Huawei Locomotion (SHL) dataset, outperforming existing single-model methods.
Introduction
The ubiquity of smartphones makes them ideal for tracking daily activities and transportation modes. However, building robust HAR models must overcome three primary obstacles:
- Long-range temporal dependencies: High sampling rates (e.g., 100Hz) create long sequences where standard RNNs or CNNs often lose context.
- Large intra-class variance: Sensor signals for the same activity (e.g., walking) vary significantly depending on whether the phone is in a pocket, bag, or hand.
- Small inter-class variance: Activities like riding a train versus a subway produce nearly identical sensor profiles, making them difficult to distinguish.
Proposed Methodology
The proposed framework addresses these issues through a multi-stage pipeline designed to handle long sequences and generalize across different usage scenarios.
Preprocessing and Feature Extraction
Raw sensor data is first "derotated" using quaternion orientation data to align accelerometer and magnetometer readings with a global North-East-Down coordinate system. This ensures consistency regardless of the phone's physical orientation relative to the user.
To manage the high-frequency data, the system segments 5-second windows into 21 overlapping 1-second sub-windows. For each sub-window, the model extracts:
- Spatial Features: Statistical measures like mean and variance.
- Frequency Features: FFT amplitude spectrums to capture rhythmic movement patterns.
- Temporal Differences: Augmenting features with changes over time to capture motion dynamics.
Long-Term Modeling with Dense IndRNN
The sequence of extracted features is processed by a deep, dense Independently Recurrent Neural Network (IndRNN). Unlike traditional LSTMs, IndRNN neurons are independent within a layer, which mitigates gradient vanishing/exploding issues and allows for much deeper architectures. These layers are organized into "Dense blocks" to maximize feature reuse across temporal scales.
Location-Aware Transfer Learning
To handle unknown phone placements and user-specific movement styles, the framework employs a two-step post-processing strategy. First, it classifies the phone's location into broad groups (e.g., Hand/Bag vs. Hips/Torso). Second, it uses fused transfer learning to fine-tune the model on a small subset of target-user data, significantly improving generalization.
Experimental Results
The framework was evaluated on the Sussex-Huawei Locomotion (SHL) dataset, which includes 8 activity classes across 4 body locations.
The Dense IndRNN framework achieved an 80.72% F1-score, surpassing single-model baselines like XGBoost (77.9%) and CNN+LSTM hybrids (52.8%). Ablation studies confirmed that the dense architecture improved performance by 2.3% over a standard IndRNN, while feature augmentation added another 2.2%. Transfer learning proved critical for user independence; for instance, accuracy for the "Run" class jumped from 43% to 94% after fine-tuning.
Conclusion
By combining short-term frequency analysis with deep long-term recurrent modeling, this framework effectively bridges the gap between raw sensor data and complex human behaviors. While it excels at distinguishing physical locomotion, future work will focus on better differentiating motor-powered transport modes (e.g., bus vs. train) to further close the performance gap with heavy ensemble methods while maintaining a single core architecture.
Summary updated