Skeleton-Based Transformer for Classification of Errors and Better Feedback in Low Back Pain Physical Rehabilitation Exercises

Aleksa Marusic, Sao Mai Nguyen, Adriana Tapus

TL;DR

This paper introduces a novel skeleton-based Transformer model for classifying errors in low back pain rehabilitation exercises. By leveraging self-attention mechanisms, the model achieves state-of-the-art performance in identifying subtle movement errors, offering more detailed feedback than existing methods and paving the way for improved patient rehabilitation.

What Problem Does This Paper Address?

Physical rehabilitation exercises are crucial for recovery from musculoskeletal disorders, but patient adherence and proper execution are significant challenges, especially in home-based settings without direct supervision. While current automated systems can assess exercise quality, they often provide only a binary "correct/incorrect" classification or a general score. This limited feedback is insufficient for patients to understand why their performance is suboptimal and how to improve. This paper addresses the need for a more granular approach that can not only detect errors but also classify them, providing actionable insights for patients and clinicians.

Methods

The core of this work is a Skeleton-Based Transformer model inspired by the HyperFormer architecture, adapted for the specific task of error classification in rehabilitation. Instead of distinguishing between different actions, this model focuses on subtle variations within a single exercise.

Key methodological aspects include:

  • Skeleton Representation: Human pose estimation is used to extract joint coordinates, forming a skeleton. This skeleton is treated as a graph.
  • Hypergraph Construction: The skeleton graph is further divided into custom "hypergraphs" (groups of joints) to capture more precise spatial relationships. The authors specifically group joints like forearms/hands, legs, and spine/head, giving particular importance to arm joints due to their role in the studied exercises.
  • Self-Attention Mechanism: Inspired by Transformers in NLP, self-attention is employed to learn relationships between joints over time and space. This allows the model to weigh the importance of different joints and their interactions for accurate error detection.
  • Multi-Head Self-Attention (MHSA): This extension allows the model to learn diverse spatial relationships in parallel.
  • Error Classification: The model is trained to classify movements into one of four categories: correct, error1, error2, or error3, specific to each exercise.
  • Joint Importance Visualization: The model's attention weights are analyzed to identify which joints are most crucial for performing each exercise correctly, providing insights into biomechanical importance.

Datasets

The primary dataset used is the Keraal dataset. This dataset is notable because it is one of the few medical datasets with detailed error labels for rehabilitation exercises. It was collected during a clinical trial for low back pain patients and includes recordings from healthy subjects and patients, annotated by physiotherapists for correctness and specific error types. The dataset contains three exercises: torso rotation, flank stretch, and hiding face.

Key Insights and Ideas

  • Beyond Binary Classification: The paper strongly advocates for moving beyond simple "correct/incorrect" feedback. Classifying specific error types is crucial for effective patient learning and rehabilitation progress.
  • Transformer Power for Subtle Differences: The Transformer architecture, particularly its self-attention mechanism, proves highly effective at capturing the subtle spatio-temporal differences that distinguish correct movements from specific errors.
  • Biomechanics Informing AI: The approach of visualizing joint importance based on attention weights offers a direct link between AI model outputs and biomechanical principles, providing interpretable feedback.
  • Data Scarcity is a Challenge: The limited availability of annotated medical data is a significant hurdle, impacting model training and evaluation, especially for underrepresented error classes.

Key Results

The proposed skeleton-based Transformer model achieves state-of-the-art performance on the Keraal dataset, outperforming previous benchmark methods in accuracy for error classification.

  • In Scenario 1 (trained on simulated errors, tested on unseen data), the model achieved accuracies of 73.17% for Torso rotation, 64.44% for Flank stretch, and 74.28% for Hiding face, significantly surpassing LSTM and GMM baselines.
  • The model demonstrates improved performance when trained on a larger, combined dataset (Scenario 2), confirming the benefit of more training data.
  • The joint importance visualization revealed that arm joints are critical for all exercises, while leg joints are less so, aligning with the nature of the rehabilitation movements. Different exercises also highlighted specific arm joints (e.g., shoulders and elbows for torso rotation and hiding face).

Limitations and Future Work

The authors acknowledge several limitations and suggest future research directions:

  • Limited Data: The Keraal dataset, while valuable, is still limited in size, particularly for certain error classes. This imbalance makes evaluation challenging and can hinder model generalization.
  • Data Augmentation: Exploring data augmentation techniques is suggested to improve model performance, especially for underrepresented classes.
  • Temporal Localization: The current model classifies entire movements. Future work could focus on temporally localizing errors within a movement sequence.
  • More Precise Instructions: Providing more specific instructions on how to fix identified errors is a key area for future development to enhance feedback quality.
  • Integrated Exercise Detection: The current model assumes the exercise type is known. A more comprehensive system could integrate exercise detection with error classification.

Summary updated

BibTeX
@inproceedings{marusic2025skeleton,
  author = {Aleksa Marusic and Sao Mai Nguyen and Adriana Tapus},
  title = {{Skeleton-Based Transformer for Classification of Errors and Better Feedback in Low Back Pain Physical Rehabilitation Exercises}},
  booktitle = {ICORR},
  year = {2025},
  doi = {10.1109/icorr66766.2025.11063192},
}

Loading related content...

Figure