TL;DR
This paper tackles the challenge of recognizing 24 daily living activities by integrating data from binary sensors, an intelligent floor, BLE beacons, and acceleration sensors. While achieving high accuracy on training data, the model struggled with unseen test data, highlighting the critical need for more labeled data and potentially refined feature engineering to bridge the gap between lab performance and real-world application.
What Problem Does This Paper Address?
Human Activity Recognition (HAR) is a well-established field, but understanding the interplay between what activities we perform and where we perform them (Indoor Localization) has been less explored. This paper aims to bridge this gap by leveraging multi-modal sensor data to recognize a comprehensive set of 24 daily living activities. The authors highlight that while HAR can improve indoor localization accuracy, and vice-versa, the lack of integrated sensing technologies previously hindered such combined approaches. Modern ubiquitous sensors, however, now make this feasible, leading to the creation of datasets like the one used here, which captures both movement and location data simultaneously.
Methods
The study employs the Cross-Industry Standard Process for Data Mining (CRISP-DM) methodology, a structured approach for data mining projects. The core of their method involves:
- Data Synchronization: Aligning data from four distinct sources (binary sensors, intelligent floor, BLE beacons, and acceleration sensors) based on timestamps, with the highest frequency sensor (acceleration) serving as the reference. Missing data points were imputed using the average of the preceding 50 samples.
- Feature Extraction: Segmenting synchronized data into 5-second windows. For each segment, 87 features were extracted, including:
- Binary sensor states (presence of "Open," "Movement," or "Pressure").
- Intelligent floor capacitance (presence of activity on a module).
- Proximity data from BLE beacons (IDs of the four nearest beacons).
- Statistical features (mean, median, standard deviation, mean absolute deviation) from 3D acceleration data.
- Modeling: Utilizing several machine learning algorithms, including J48, Ib1, SVM, Random Forest (RF), AdaBoostM1 (ABM1), and Bagging. The J48 algorithm was used as the base classifier for ensemble methods. A Java desktop application, adapted from previous work, was used for training and evaluation.
- Activity Merging: A key insight was merging similar activities (e.g., "prepare breakfast, lunch, and dinner" into one, and "breakfast, lunch, and dinner" into another) to improve classification accuracy, as these activities often exhibit similar sensor patterns.
Datasets
The primary dataset used is from the 1st UCAmI cup. This dataset was collected in the UJAmI SmartLab and contains data for 24 activities of daily living performed over 10 days. It comprises:
- Part One (Training Data): Seven days of labeled recordings.
- Part Two (Test Data): Three days of unlabeled recordings.
The dataset is unique in its integration of four data sources: binary sensors, an intelligent floor, BLE beacons, and acceleration sensors, specifically designed to capture the relationship between HAR and indoor localization.
Key Insights and Ideas
- The Power of Multi-Modality: Combining data from various sensors (binary, floor, BLE, acceleration) is crucial for a comprehensive understanding of human activities.
- Activity Granularity vs. Sensor Data: The observation that merging similar activities significantly improved accuracy suggests that the sensor data might not always be distinct enough to differentiate highly similar actions. This points to a potential limitation in the sensor resolution or the need for more sophisticated feature engineering.
- The "Idle" Activity: The "Idle" state, representing periods with sensor data but no labeled activity, was found to be a source of misclassification. The authors suggest that in real-world deployments, this category might be unnecessary, potentially boosting accuracy by its exclusion.
- Class Imbalance is a Major Hurdle: The significant drop in accuracy from training to testing data strongly suggests that class imbalance in the training set is a critical issue. Activities with fewer examples are harder for the model to learn effectively.
Key Results
- Training Accuracy: Using 10-fold cross-validation on the labeled training data (Part One), the best performing algorithm, AdaBoostM1, achieved an accuracy of 92.1%.
- Testing Accuracy: On the unseen test data (Part Two), the model achieved a significantly lower accuracy of 60.1%.
- Specific Misclassifications: The confusion matrix revealed that "idle," "Watching TV," and "Playing video game" were frequently misclassified, often as "Relax on the sofa."
- Excluding "Idle": If the "Idle" category were excluded, the test accuracy would increase to 62.77%.
Limitations and Future Work
- Low Test Accuracy: The primary limitation is the substantial drop in accuracy on the test dataset compared to the training set. The authors attribute this to potential class imbalance in the training data.
- Need for More Labeled Data: To address the class imbalance and improve generalization, the authors explicitly state that more labeled data is necessary for training the algorithm.
- Further Analysis of Misclassified Activities: The misclassification of "Watching TV" and "Playing video game" warrants deeper investigation. Future work should focus on analyzing these specific cases to identify additional features or rules that could improve their classification.
- Iterative Refinement: The authors acknowledge that the current results are not optimal and suggest iterating within the CRISP-DM methodology to refine the approach and improve accuracy.
- Real-World Deployment Considerations: For practical applications, rules based on the time of day and activity sequences could be incorporated to enhance classification, especially for activities like "prepare breakfast, lunch, and dinner."
Summary updated