Introduction
Human Activity Recognition (HAR) using wearable sensors is a cornerstone of modern health monitoring and human-computer interaction. Traditionally, HAR relied on manually engineered features—statistical measures like mean, variance, or spectral energy—to interpret sensor data. However, these "hand-crafted" features often struggle to capture the complex, non-linear patterns of human movement.
While Deep Learning (DL) has emerged as a powerful alternative, the field has lacked a systematic comparison of different architectures. Many studies report only "peak performance" without detailing hyperparameter sensitivity or robustness. This paper addresses this gap by conducting over 4,000 experiments to compare Deep Feed-forward (DNN), Convolutional (CNN), and Recurrent (RNN) models across diverse HAR benchmarks.
Architectures and Methodology
The authors evaluated four primary architectural configurations to identify the most effective approach for varying activity types:
- Deep Feed-forward Networks (DNN): Standard multi-layer perceptrons that process flattened sensor data frames.
- Convolutional Neural Networks (CNN): Designed to capture local temporal patterns through shared weights, making them invariant to the exact timing of a movement within a window.
- Recurrent Neural Networks (LSTM): Specifically Long Short-Term Memory cells, which model temporal dependencies. The study explored both unidirectional (Forward) and Bi-directional LSTMs.
- Novel Regularization: To improve generalization, the authors introduced a "carry-over probability" ($p_{carry}$). This technique randomly resets the internal state of the RNN during training, preventing the model from over-relying on long-term sequences that might not generalize to new subjects.
Experimental Setup
The models were tested on three benchmark datasets:
- Opportunity (Opp): Focuses on short, non-repetitive gestures like "opening a drawer."
- PAMAP2: Contains prolonged, repetitive physical activities such as "Nordic walking."
- Daphnet Gait (DG): A medical dataset targeting "Freezing of Gait" in Parkinson’s patients.
To rigorously analyze performance, the researchers utilized the fANOVA framework. This statistical method decomposes the variance in model performance to determine which hyperparameters—such as learning rate, number of layers, or filter sizes—most significantly impact the final F1-score.
Results and Discussion
The systematic comparison revealed distinct strengths and weaknesses for each architecture.
Recurrent Models and Complex Gestures
The bi-directional LSTM (b-LSTM-S) emerged as the top performer for complex gesture recognition. On the Opportunity dataset, it achieved a mean F1-score of 0.745, significantly outperforming the previous state-of-the-art CNN models (0.704). This highlights the importance of temporal context in identifying subtle, non-repetitive movements.
CNNs as the Robust Choice
While LSTMs reached the highest performance peaks, CNNs proved to be the most consistent. As shown in the performance distribution plots, CNNs exhibited less variance across different hyperparameter settings. This makes them a reliable "out-of-the-box" choice for practitioners who may not have the resources for extensive hyperparameter tuning.
The Sample-by-Sample Advantage
A significant contribution of this work is the demonstration of "sample-by-sample" prediction using LSTMs. Traditional HAR systems use sliding windows, which inherently introduce a processing delay. LSTMs can process data point-by-point, enabling near-zero-latency activity tracking—a critical requirement for real-time applications like gait assistance or interactive gaming.
Conclusion
This study provides a definitive benchmark for deep learning in HAR. It establishes that while RNNs (specifically bi-directional LSTMs) are superior for complex gestures and real-time applications, CNNs offer a more stable and easily tunable alternative for general tasks. The findings suggest that future research should focus on optimizing deeper architectures for sensor data and further exploring the real-time potential of sample-by-sample recurrent modeling.
Summary updated