Wrist movement classification for adaptive mobile phone based rehabilitation of children with motor skill impairments

Kayleigh Schoorl, Tamara V. Pinos Cisneros, Albert Ali Salah, Ben A. M. SchoutenarXiv 2024 · DOI · OpenReview

Abstract

Researchers have developed "Magic Monster," a mobile exergame designed to transform tedious wrist rehabilitation for children with cerebral palsy (CP) into an engaging magic-themed challenge. By leveraging standard smartphone accelerometers and gyroscopes, the system utilizes a Convolutional Neural Network (CNN) to accurately classify therapeutic movements. Crucially, the system automatically adjusts difficulty to match each child's unique range of motion, ensuring both accessibility and therapeutic progress.

Introduction

For children with cerebral palsy, physical therapy is a lifelong necessity that often becomes repetitive and unmotivating. Exercises such as dorsiflexion—bending the wrist upward—are vital for maintaining motor skills but are frequently neglected due to their boring nature. This research addresses the dual challenge of increasing engagement through gamification and ensuring exercise accuracy through automated movement classification. Because every child with CP has a different baseline range of motion, a standard detection system is insufficient; an adaptive approach is required to meet the child where they are.

System Design: The Magic Monster

The researchers developed a mobile application housed within a monster-shaped plush sheath. The smartphone screen serves as the monster's face, providing immediate visual feedback as the child performs "magic tricks" via wrist movements.

The Magic Monster interface, showing the physical plush sheath and the digital game screen.

This physical interface makes the device less intimidating and provides a better grip for children with limited dexterity. The system includes a five-movement calibration phase, allowing the game to adapt to the specific physical capabilities of the user rather than enforcing a rigid standard.

Data Processing and Classification

The movement recognition engine relies on data from the phone's internal accelerometer and gyroscope.

Representative accelerometer and gyroscope data captured during wrist movement exercises.

The team extracted 42 statistical features (including mean, variance, and skew) from the three axes of both sensors. Using the Minimum Redundancy Maximum Relevance (mRMR) algorithm, they identified the 21 most effective features. Notably, the standard deviation of the gyroscope's x-axis emerged as the strongest indicator for range of motion.

While Long Short-Term Memory (LSTM) networks are often the standard for time-series data, this study found that a Convolutional Neural Network (CNN) achieved superior performance. The CNN's spatial filters were better suited to capturing the specific "burst" patterns characteristic of wrist dorsiflexion.

Results and Adaptive Difficulty

The CNN model achieved an impressive accuracy of 0.970 on independent test sets and maintained a high accuracy of 0.948 during field tests with children with CP. The system demonstrated higher precision (0.929) than recall (0.846), ensuring that children are not incorrectly credited for improper movements.

To maintain motivation, the system employs a rule-based adaptive difficulty mechanism. By monitoring the last 10 movements, the game increases the required range of motion or speed if the success rate exceeds 90%, or lowers the threshold if it falls below 60%.

Conclusion and Future Work

Magic Monster demonstrates the potential for low-cost, sensor-based mobile games to support home-based rehabilitation. While the current results are promising, the study's small sample size (8 children) and reliance on healthy adult data for initial training suggest areas for improvement. Future research will focus on longitudinal studies to evaluate long-term engagement and the impact of sensor hardware variations across different smartphone models.

Summary updated

Abstract

Rehabilitation exercises performed by children with cerebral palsy are tedious and repetitive. To make them more engaging, we propose to use an exergame approach, where an adaptive application can help the child remain stimulated and interested during exercises. In this paper, we describe how the mobile phone sensors can be used to classify wrist movements of the user during the rehabilitation exercises to detect if the user is performing the correct exercise and illustrate the use of our approach in an actual mobile phone application. We also show how an adaptive difficulty system was added to the application to allow the system to adjust to the user. We present experimental results from a pilot with healthy subjects that were constrained to simulate restricted wrist movements, as well as from tests with a target group of children with cerebral palsy. Our results show that wrist movement classification is successfully achieved and results in improved interactions.
BibTeX
@misc{schoorl2024wrist,
  author = {Kayleigh Schoorl and Tamara V. Pinos Cisneros and Albert Ali Salah and Ben A. M. Schouten},
  title = {{Wrist movement classification for adaptive mobile phone based rehabilitation of children with motor skill impairments}},
  year = {2024},
  eprint = {2401.17134},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2401.17134},
}
Figure