TL;DR
This paper presents a novel platform for real-time human activity recognition by combining sensor data from smartphones and smartwatches. The authors demonstrate that this integrated approach significantly improves accuracy compared to using a smartphone alone, with Naive Bayes emerging as the most efficient and accurate classifier.
What Problem Does This Paper Address?
Accurately and unobtrusively monitoring human physical activity is crucial for various applications, including health monitoring and promoting healthier lifestyles. While smartphones and smartwatches are ubiquitous and packed with sensors, effectively leveraging this data for real-time activity recognition, especially for complex activities, remains a challenge. Existing research often relies on specialized, multi-sensor setups that are impractical for everyday use, or uses single devices with limited accuracy. This paper aims to bridge this gap by developing a practical, mobile, and accurate activity recognition system using readily available consumer devices.
Methods
The proposed system employs a multi-stage approach:
- Data Collection: Time-series data is collected from smartphone and smartwatch sensors at a 30 Hz sampling rate.
- Segmentation: A sliding window approach is used to divide the time series data into segments of fixed duration without overlap, which is ideal for real-time applications.
- Preprocessing: A median filter is applied to remove noisy values and outliers from the accelerometer data.
- Feature Extraction: Features are extracted from both time and frequency domains. This includes statistical features like Mean, Minimum, Maximum, Range, Standard Deviation, and Root-Mean-Square from the time domain. Additionally, the dominant and second dominant frequencies are extracted from the frequency domain using Fast Fourier Transform (FFT). Features are computed from the three axial accelerometer components (x, y, z) and their magnitude.
- Standardization: Z-score standardization is applied to ensure all features have a zero mean and unit variance, making them comparable for distance-based classification algorithms.
- Classification: Several classification algorithms are evaluated, including Support Vector Machines (SVM), Decision Trees, Naive Bayes, Discriminant Analysis (LDA, QDA), and K-Nearest Neighbors (KNN). Feature selection is performed using a greedy forward selection approach to identify the optimal subset of features for each classifier.
Datasets
The authors collected a new dataset from 16 participants (8 males, 8 females) performing eight activities: four everyday activities (walking, jogging, idle, stair climbing) and four fitness exercises (rope jumping, pushups, crunches, squats). The data was collected while participants wore a smartwatch on their left hand and carried a smartphone in their front right pocket. The dataset was segmented into five different subsets using window durations of 1, 2, 3, 4, and 5 seconds for evaluation.
Key Insights and Ideas
- Synergy of Devices: Combining sensor data from both smartphones and smartwatches significantly boosts activity recognition accuracy, suggesting a powerful synergy between these wearable devices.
- Naive Bayes Efficiency: Naive Bayes, despite its simplicity, proves to be a highly effective classifier, offering a strong balance between computational efficiency and recognition accuracy.
- Feature Importance: A carefully selected subset of 18 features, derived from both time and frequency domains, is sufficient for achieving high accuracy, demonstrating that not all available sensor data is equally informative.
- Sampling Frequency Trade-off: While higher sampling frequencies (like 30 Hz) capture more detail, a lower frequency of around 10 Hz can still yield comparable accuracy with reduced computational load and memory usage.
Key Results
- Overall Accuracy: The proposed platform achieved an overall classification accuracy of 87% ± 2.4%.
- Best Classifier: Naive Bayes achieved the highest average accuracy of 89.4% using an 18-feature subset.
- Smartphone vs. Combined: Using only smartphone data, the highest accuracy achieved was 82.1% (with Naive Bayes). Adding smartwatch data improved this by at least six percentage points.
- Sampling Frequency: An average accuracy of 88.8% was achieved at a sampling frequency of 10 Hz, a slight decrease from 30 Hz but with potential resource savings.
- Confusion: All tested classifiers struggled to differentiate between "walking" and "stair climbing" activities.
Limitations and Future Work
- Dataset Size: The authors acknowledge that larger experiments with more participants are needed for more robust evaluation and to definitively determine if one method is superior or if off-the-shelf methods are sufficient.
- Activity Scope: The current system focuses on a limited set of activities. Future work could extend this to recognize higher-level activities like shopping or eating dinner.
- Sensor Integration: Incorporating additional sensors, such as heart rate monitors, could further enhance recognition accuracy.
- Generalization: Investigating methods to extrapolate trained classifiers to new users without requiring extensive individual training data is a promising direction.
Summary updated