Abstract
Human activity recognition (HAR) provided several benefits to people without disabilities. Researchers have provided public HAR datasets and developed technologies based on these datasets. However, the activities of pedestrians with mobility disabilities have not been actively investigated because related datasets do not exist. In this study, we compile an HAR dataset comprising basic activities for people with and without mobility disabilities (including activity categories of still, walking, crutches, walkers, manual wheelchairs, and electric wheelchairs). Our dataset contains sensor data from smart devices (smartphones and smartwatches) collected from 120 participants. We also provide baseline analyses of our dataset: (1) recognition tasks according to the pedestrian activities, (2) impact of sensor combinations, (3) classification models, (4) evaluation methods, and (5) combining smartphone and smartwatch sensors. The results indicate that the classification accuracies are 99.64% for the random evaluation and 98.79% for the user-independent evaluation using the best combinations. We hope that this study will expand HAR research for people with mobility disabilities and renew enthusiasm for subsequent applications related to this topic.
Introduction
Human Activity Recognition (HAR) has become a cornerstone of modern mobile technology, enabling applications ranging from fitness tracking to health monitoring. However, most existing public datasets focus exclusively on able-bodied movements like walking, running, or sitting. This "data gap" leaves pedestrians with mobility disabilities underserved, as current algorithms often fail to accurately recognize movements involving assistive devices.
To address this, researchers have introduced HAR-PMD, the first publicly available HAR dataset specifically designed for pedestrians using mobility aids. This dataset enables the development of context-aware services, such as accessible navigation and personalized physical therapy tools, tailored to the needs of individuals with mobility challenges.
The HAR-PMD Dataset
The HAR-PMD (Human Activity Recognition for Pedestrians with Mobility Disabilities) dataset consists of high-resolution sensor data collected from 120 participants.
- Activities: The dataset covers six distinct categories: Still, Walking, Crutches, Walker, Manual Wheelchair, and Electric Wheelchair.
- Sensors: Data was captured using 13 smartphone sensors and 5 smartwatch sensors, including linear acceleration, gyroscopes, and magnetometers.
- Environment: To ensure real-world applicability, data collection occurred in both indoor (hospitals/offices) and outdoor (streets/roads) settings.
Methodology
The study employed a comprehensive pipeline for data processing and model evaluation:
- Preprocessing: Sensor data was interpolated to a consistent 60 Hz and segmented into 5-second windows.
- Feature Engineering: For classical machine learning, 11 time- and frequency-domain features were extracted per axis.
- Model Benchmarking: The researchers compared classical models (Random Forest, XGBoost, SVM) against deep learning architectures (CNN, LSTM, Transformers).
- Evaluation Protocols: Models were tested using both User-Dependent (random split) and User-Independent (leave-one-group-out) methods to assess generalization to new users.
Key Results and Insights
The study yielded several significant findings regarding the recognition of mobility-aid-assisted activities:
- High Accuracy: The best-performing models achieved a peak accuracy of 99.64% in user-dependent tests and 98.79% in user-independent scenarios.
- Multi-Device Fusion: A critical insight was the value of synchronized smartwatch data. While smartphones alone struggle to distinguish between similar upper-body movements (e.g., crutches vs. walkers), wrist-based sensors provide the necessary motion data to disambiguate these activities.
- Sensor Combinations: The combination of Linear Accelerometer, Gyroscope, and Magnetometer (AGM) consistently outperformed single-sensor configurations.
- Model Performance: While deep learning models performed well, the XGBoost ensemble model often matched or exceeded their performance, demonstrating the continued relevance of expert feature engineering in sensor-based HAR.
Conclusion and Future Work
The release of HAR-PMD marks a significant step toward inclusive HAR technology. While the current dataset provides a strong baseline, future work should focus on collecting data from long-term users with actual mobility disabilities to capture authentic usage patterns. Additionally, expanding the demographic to include older adults and moving toward "in-the-wild" data collection will be essential for deploying these models in real-world assistive technologies.
Summary updated