TL;DR
This paper investigates optimal system configurations for physical activity recognition (PAR) on smartwatches, a field still in its early stages. By analyzing various feature extraction methods and classification algorithms, the authors identify key settings that enhance PAR performance for both personal (user-specific) and impersonal (general) models, paving the way for more accurate and reliable smartwatch-based health monitoring.
What Problem Does This Paper Address?
While recognizing physical activities using smartphones has been extensively researched, the application of this technology to smartwatches is relatively nascent. Smartwatches, with their embedded sensors like accelerometers, gyroscopes, and magnetometers, offer a promising platform for continuous, unobtrusive health monitoring. However, the unique form factor and sensor placement on a smartwatch (wrist vs. pocket/hip) might necessitate different approaches to feature extraction and classification compared to smartphones. This paper aims to bridge this gap by systematically exploring and optimizing the system settings for smartwatch-based PAR, addressing the question of which features and algorithms yield the best results for both individual users and generalizable models.
Methods
The authors conducted a large-scale exploratory study to identify optimal system settings for smartwatch-based PAR. Their methodology involved:
- Sensor Data: Utilizing data from the accelerometer, magnetometer, and gyroscope sensors embedded in smartwatches.
- Feature Extraction: Extracting features from raw sensor data using two distinct banks:
- Feature Bank 1: Statistical features like mean, median, variance, standard deviation, autocorrelation, and coefficients from autoregressive and moving average models.
- Feature Bank 2: Features related to acceleration, including average acceleration, standard deviation of resultant acceleration, and peak differences.
- They also explored binned distributions of sensor readings.
- Classification: Evaluating five common machine learning classifiers: Decision Tree, Naive Bayes, K-Nearest Neighbors (KNN), Support Vector Machine (SVM), and Ensemble Learning (BAG). Artificial Neural Networks (ANNs) were excluded from this study.
- Model Types: Investigating two types of models:
- Personal Model: Trained and tested on data from a single user.
- Impersonal Model: Trained on data from all users except the one being tested.
- Experimental Variations: Systematically varying parameters such as:
- Feature Normalization and Permutation: Assessing the impact of normalizing features and randomly permuting them.
- Window Size: Analyzing the effect of different window sizes (from 25 to 300 samples) on recognition accuracy.
- Cross-Validation: Employing a 10-fold cross-validation strategy and a single-subject cross-validation process to ensure robust evaluation.
Datasets
The paper utilized a custom dataset collected from six users (three female, three male) with a mean age of twenty-five. Participants performed five distinct activities: walking, walking upstairs, walking downstairs, running, and jogging. Each subject performed these activities twice daily for over a month, with data collected using a smartwatch and a paired smartphone. The raw sensor data was recorded at a 20Hz rate. The dataset is not explicitly made publicly available in the paper, but the authors note that it is not perfectly balanced, which they addressed using cross-validation techniques.
Key Insights and Ideas
- Smartphone Features Translate: Features commonly used for smartphone-based PAR are indeed effective for smartwatch-based PAR, suggesting a degree of transferability in feature engineering.
- Normalization and Permutation Matter: Normalizing features and randomly permuting them significantly improves classification accuracy across various classifiers and activities, especially for impersonal models. This highlights the importance of data preprocessing for robust performance.
- Personal vs. Impersonal Models: Personal models generally achieve higher accuracy than impersonal models, which is expected as they are tailored to individual user patterns. However, the study also demonstrates that well-configured impersonal models can achieve respectable accuracy.
- Window Size Impact: The choice of window size has a notable impact on recognition accuracy. While smaller windows generally lead to better performance, especially for KNN and SVM, Decision Trees show less degradation with larger window sizes.
- Classifier Performance Hierarchy: Decision Trees and SVM consistently emerged as top performers, often outperforming other classifiers with smaller confidence intervals, suggesting greater reliability.
Key Results
- High Accuracy Achieved: The study reports high accuracies, with personal models achieving up to 98% accuracy and impersonal models reaching 94% and 90% with optimized settings.
- Best Settings Identified: The recommended optimal system settings for smartwatch-based PAR are:
- Features: 70 features extracted from the sensor data.
- Preprocessing: Features should be normalized and randomly permuted.
- Classifiers: Decision Tree and SVM classifiers are recommended.
- Window Size: A window size between 25 and 125 samples per window is generally optimal.
- Decision Tree Dominance: Decision Tree classifiers consistently showed superior performance across various activities and model types, particularly when combined with normalized and randomly permuted features.
Limitations and Future Work
- Dataset Imbalance: The authors acknowledge that their dataset is not perfectly balanced, which could lead to biased results. While they mitigated this with cross-validation, a more balanced dataset would further improve robustness.
- Limited Activities: The study focused on five specific activities. Future work could explore a wider range of activities, including more complex or subtle movements.
- Feature Selection Methods: The paper suggests investigating linear/non-linear supervised and unsupervised feature selection methods in future work to further optimize feature extraction.
- Real-time Performance: While the study focuses on optimal settings, real-time implementation and computational efficiency were not the primary focus and could be explored further.
Summary updated