Introduction
Human Action Recognition (HAR) has traditionally relied on "skeleton data"—sequences of 3D coordinates representing joint positions. While effective, this representation suffers from viewpoint dependency, subject-specific variations (like height), and a lack of semantic interpretability. This paper proposes a shift toward standardized biomechanical joint angles based on International Society of Biomechanics (ISB) standards. By representing motion through viewpoint-independent, human-interpretable angles, the authors aim to create models that are more robust and efficient.
The Case for Biomechanical Standards
The biomechanics community has utilized rigorous standards for describing human motion since the 1980s. In contrast, the computer vision community often uses raw XYZ coordinates or simple dot-product angles. The authors argue that adopting Joint Coordinate Systems (JCS) can address several fundamental issues:
- Viewpoint Independence: Coordinates change with camera movement, but joint angles relative to the body do not.
- Subject Normalization: Differences in limb length are naturally factored out when focusing on rotation.
- Interpretability: Experts like clinicians or coaches describe movement in terms of flexion, extension, or abduction, not Cartesian coordinates.
Methodology: From Keypoints to Joint Coordinate Systems
The core contribution is a systematic conversion of keypoint data into JCS. Following ISB recommendations, local coordinate systems are defined for each joint, decomposing motion into three primary planes: sagittal (flexion/extension), frontal (abduction/adduction), and transverse (internal/external rotation).
Unlike simpler angle representations, these biomechanical angles provide a nearly bijective mapping. Given known bone lengths, the original skeleton can be perfectly reconstructed from the angles. To ensure performance gains were due to the data representation rather than specific architectures, the authors tested the approach using standard 1D Convolutions, LSTMs, and Transformers.
Results and Discussion
A key finding was the "Hard" Task Advantage. On the Skeletics "Hard" subset, joint angles notably outperformed raw keypoints, capturing the subtle nuances required to distinguish between similar actions, such as different types of ball throws.
Robustness to Rotation
In stress tests involving random input rotations, keypoint-based models saw accuracy drop significantly (e.g., from 82.9% to 47.3%). In contrast, joint-angle models maintained perfectly stable performance, as the representation is mathematically rotation-invariant by design.
Data Efficiency and Ensembling
Joint angles represent motion using approximately 1/3 of the data required for 3D keypoints. Furthermore, the best overall results were achieved by ensembling keypoint-based and angle-based models. On NTU RGB+D, an ensemble reached 84.4% accuracy, outperforming individual keypoint (82.9%) and angle (70.6%) models.
Conclusion and Future Work
While joint angles show immense promise for interpretability and robustness, they exhibit sensitivity to noise in low-quality data (like Kinect). Future work includes optimizing architectures like Graph Convolutional Networks (GCNs) for joint angle structures and expanding datasets to include the additional keypoints required for full ISB standard implementation. The authors have released a Python package to facilitate the conversion of keypoint data into these standardized biomechanical angles, bridging the gap between computer vision and biomechanics.
Summary updated