TL;DR
This paper investigates how to best train activity recognition (AR) systems using smartphone data for stroke survivors, finding that training on data from stroke patients themselves, rather than healthy individuals, significantly improves performance. Crucially, it highlights that AR models trained in a lab setting perform poorly when applied to real-world home activities, suggesting the need for at-home data in training.
What Problem Does This Paper Address?
Monitoring the physical activity of stroke survivors is vital for understanding their recovery and the effectiveness of rehabilitation. Smartphones, with their built-in sensors, offer a promising, unobtrusive way to collect this data. However, existing AR systems are often trained on data from healthy individuals in controlled lab environments. This approach fails to account for the unique movement patterns of stroke survivors, particularly those with gait impairments, and the different contexts in which activities are performed at home versus in a lab. This disconnect leads to inaccurate activity recognition, hindering the development of effective, personalized rehabilitation strategies. The paper aims to bridge this gap by evaluating AR performance for stroke patients under various training conditions.
Methods
The researchers employed a Random Forest (RF) model, a standard machine learning algorithm known for its efficiency and accuracy in classification tasks, particularly with large feature sets. To handle potential class imbalance (e.g., fewer instances of stair climbing compared to sitting), they incorporated the RUSBoost algorithm, which combines random undersampling with boosting to improve performance on underrepresented classes.
The core of their methodology involved comparing different training and testing scenarios:
- Population Models: Training AR models on data from either healthy individuals or stroke patients and testing them on stroke patients.
- Gait Impairment Models: Evaluating how well a healthy-trained model performs on stroke patients with varying degrees of gait impairment (mild, moderate, severe).
- Environmental Models: Comparing the performance of models trained in a lab setting versus those trained at home, when tested on activities performed at home.
They collected data using a custom app on Samsung Galaxy S4 smartphones, capturing accelerometer, gyroscope, and barometer data. Participants self-labeled their activities, and the data was transmitted to a secure server. Features extracted from the sensor data included statistical measures, derivatives, and power spectra.
Datasets
The study utilized two primary datasets:
- Healthy Subject Data: Collected from 15 healthy individuals.
- Stroke Subject Data: Collected from 30 individuals who had suffered a stroke, with varying levels of gait impairment.
This data was collected in both laboratory and home settings. The paper does not introduce a new benchmark dataset but rather uses existing data collection methods and applies them to a specific population (stroke survivors) in different environments to evaluate AR performance.
Key Insights and Ideas
- "Stroke Knows Stroke Best": Training AR models on data from stroke survivors themselves is significantly more effective than using data from healthy individuals. This is likely due to the distinct movement patterns associated with stroke, such as gait abnormalities.
- The Lab vs. Home Divide: AR models trained solely in a laboratory setting perform poorly when applied to activities performed in a home environment. This highlights the critical need to incorporate real-world, at-home data into training sets for accurate community monitoring.
- Gait Impairment Matters: The severity of gait impairment in stroke survivors directly impacts the performance of AR models trained on healthy data. Models trained on healthy individuals struggle to accurately classify ambulatory activities in those with severe impairments, often mistaking them for stationary ones.
- Barometer's Niche Role: While accelerometers and gyroscopes are the primary sensors for AR, the barometer showed a notable improvement in recognizing stair-related activities, suggesting its utility for specific movement classifications.
Key Results
- Stroke-trained models achieved a mean recall of 75%, a significant improvement over models trained on healthy data, which yielded only 53% recall when tested on stroke patients.
- The healthy-trained classifier's performance declined with increasing gait impairment severity. For ambulatory activities, recall dropped from 56% in mild impairment to 23% in severe impairment.
- Models trained on lab activities had a lower average recall for at-home activities (56%) compared to models trained on in-lab activities tested on a different day (77%).
- The barometer sensor improved the recognition of stair activities, reducing misclassification of "Stairs Up" as "Walking" from 31.32% to 13.12%.
Limitations and Future Work
The authors acknowledge several limitations:
- Age Mismatch: The healthy cohort was not age-matched to the stroke cohort, which might have underestimated the performance of the healthy-trained model. However, they argue that reduced walking speeds in stroke survivors, regardless of age, would still lead to performance differences.
- Phone Placement: Placing the phone on the waist might limit the ability to distinguish between sitting and standing compared to pocket placement. This was a deliberate choice to minimize user discomfort and maintain consistent access.
- Limited Home Data: The study had a reduced amount of home data for validation due to transmission issues (poor 4G/LTE signals) and protocol design. Stroke subjects only completed one at-home labeling session, potentially overestimating the performance of Home-to-Home models.
Future work should focus on:
- Collecting more diverse at-home data from stroke survivors across various environments.
- Investigating the optimal sample size for training AR models, noting that around 10 healthy subjects and 14 stroke subjects achieved satisfactory performance with 20 minutes of data.
- Exploring the generalizability of AR models across multiple environmental contexts beyond just the lab and home.
- Developing AR models that can adapt to individual differences in gait impairment and activity performance.
Summary updated