Vision-based categorization of upper body motion impairments and post-stroke motion synergies

Babak Taati, Jennifer L. Campos, Jeremy Griffiths, Mona Gridseth, Alex MihailidisInternational Journal on Disability and Human Development 2014 · DOI

Abstract

Upper body motion impairment is a common after-effect of a stroke. We are developing a novel multisensory therapy system that makes use of augmented feedback and is expected to enhance rehabilitation outcomes. The first phase of the project involved developing algorithms to automatically differentiate between normal and impaired upper body motions. Seven healthy subjects performed two types of scripted actions: (a) elbow flexion and extension and (b) reaching over via elbow flexion and shoulder flexion and adduction. Each action was repeated 10 times as each participant felt most comfortable and also 10 times simulating a common post-stroke impaired motion according to the literature. Principal component analysis was applied to the upper body trajectories during each action, as observed with a Kinect sensor, to extract the dominant modes of motion. Three classification algorithms and up to three motion modes were examined in order to distinguish between normal and impaired motions. A statistical analysis of the Kinect skeletal tracking data vs. manual annotation confirmed a significant bias in the tracking of an elevated shoulder joint. Despite this bias, leave-one-subject-out cross-validation experiments confirmed the effectiveness of the proposed methods in detecting impaired motions (accuracy > 95%), even when the impairment involved elevating a shoulder. A single, most dominant motion mode provided sufficient information for the binary classification task. The high accuracy rates validate the use of vision-based pose tracking technologies in identifying motion deficiencies.

Introduction

Stroke survivors often experience hemiparesis, leading to the development of "motion synergies"—abnormal, involuntary couplings of muscle groups. For instance, a patient attempting to reach across their body might compensate for limited elbow mobility by hiking their shoulder or rotating their trunk. Traditional rehabilitation, such as mirror box therapy, is effective but lacks automated, objective assessment tools that are accessible outside of specialized clinics. This research explores the use of consumer-grade depth sensors and machine learning to automatically categorize these impairments, aiming to bridge the gap between clinical expertise and home-based recovery.

Methodology

The study employed a Microsoft Kinect sensor to capture 3D skeletal joint data from seven healthy subjects (4 males, 3 females). Participants performed two scripted actions: (a) elbow flexion and extension and (b) reaching across to the opposite shoulder. Each action was performed 10 times normally and 10 times while simulating common post-stroke impairments, such as shoulder hiking and trunk rotation, based on clinical literature.

Experimental setup showing the initial posture for the reaching task.

The technical pipeline involved normalizing 3D joint coordinates into a subject-centered canonical frame to ensure consistency across different body sizes and camera angles. Principal Component Analysis (PCA) was then applied to the displacement vectors of upper body joints to extract "dominant modes" of motion. These modes served as features for three classification algorithms: Logistic Regression (LR), Support Vector Machines (SVM) with radial basis function (RBF) kernels, and Random Forests (RF). To validate the sensor's accuracy, the researchers compared Kinect tracking against manual 2D annotations from human observers.

Results and Discussion

The Support Vector Machine (SVM) was the top performer, identifying simulated impaired motions with 95.7% to 97.1% accuracy. Random Forests also showed high consistency, ranging from 90% to 97.9% accuracy. In contrast, simple Logistic Regression struggled (54%–64% accuracy), highlighting that motion synergies are complex, non-linear patterns requiring sophisticated algorithmic approaches.

Subject performing the reaching across task, illustrating the type of movement analyzed for post-stroke synergies.

A key finding was the efficiency of the "Single Mode" approach; the most dominant motion mode ($n=1$) from the PCA was often sufficient for high classification accuracy. Furthermore, the study identified a significant tracking bias in the Kinect sensor, which consistently underestimated the degree of shoulder elevation (correlation $\rho \approx 0.15 - 0.20$). Despite this hardware limitation, the machine learning models remained highly accurate, likely by picking up on secondary cues such as the orientation of the shoulder-elbow segment. This robustness validates the feasibility of using affordable, vision-based systems for clinical-grade motion categorization.

Conclusion

This research demonstrates that consumer-grade depth sensors, combined with PCA and advanced machine learning, can effectively identify post-stroke motion synergies with high accuracy. While the current study utilized simulated impairments by healthy subjects, the results provide a strong foundation for future validation with actual stroke survivors. Future work will focus on integrating this system into virtual reality environments that combine "mirror box" and "rubber hand" illusions, providing real-time feedback to encourage neuroplasticity and improved motor control.

Summary updated

Abstract

Upper body motion impairment is a common
\nafter-effect of a stroke. We are developing a novel multisensory
\ntherapy system that makes use of augmented feedback
\nand is expected to enhance rehabilitation outcomes. The
\nfirst phase of the project involved developing algorithms to
\nautomatically differentiate between normal and impaired
\nupper body motions. Seven healthy subjects performed
\ntwo types scripted actions: (a) elbow flexion and extension
\nand (b) reaching over via elbow flexion and shoulder
\nflexion and adduction. Each action was repeated 10 times
\nas each participant felt most comfortable and also 10 times
\nsimulating a common post-stroke impaired motion according
\nto the literature. Principal component analysis was
\napplied to the upper body trajectories during each action,
\nas observed with a Kinect sensor, to extract the dominant
\nmodes of motion. Three classification algorithms and up to
\nthree motion modes were examined in order to distinguish
\nbetween normal and impaired motions. A statistical analysis
\nof the Kinect skeletal tracking data vs. manual annotation
\nconfirmed a significant bias in the tracking of an elevated
\nshoulder joint. Despite this bias, leave-one-subject-out cross
\nvalidation experiments confirmed the effectiveness of the
\nproposed methods in detecting impaired motions (accuracy
\n> 95%), even when the impairment involved elevating
\na shoulder. A single, most dominant motion mode provided
\nsufficient information for the binary classification task. The
\nhigh accuracy rates validate the use of vision-based pose
\ntracking technologies in identifying motion deficiencies.
BibTeX
@article{taati2014vision,
  author = {Babak Taati and Jennifer L. Campos and Jeremy Griffiths and Mona Gridseth and Alex Mihailidis},
  title = {{Vision-based categorization of upper body motion impairments and post-stroke motion synergies}},
  journal = {International Journal on Disability and Human Development},
  year = {2014},
  doi = {10.1515/ijdhd-2014-0326},
}
Figure