Motion Analysis and Transfer Applications Based on Posture Recognition

Yuhao Zhang, Mingyue Li, Jianhao JiaoProceedings of International Conference on Artificial Life and Robotics 2024 · DOI

Abstract

This paper investigates the posture trajectories in human motion using pose recognition technology based on the deep learning framework MediaPipe. By detecting key points on the human body and plotting and calculating these trajectories in the temporal dimension, we further conduct comparative analysis of these trajectories with professional sports coaches' motion guidance to assist athletes in correcting their posture. Additionally, this technology has been deployed on Jetson Nano, enabling its practical application in mobile scenarios, and providing robust tools and methods for fields such as rehabilitation therapy, sports training, and animal behavior analysis. This study offers insights into the transfer applications of posture recognition.

Introduction

Athletes training independently often lack the immediate, expert feedback required to correct their form, which can lead to plateaued performance or injury. While professional motion capture systems exist, they are typically expensive, tethered to high-end PCs, and confined to controlled laboratory environments. This research addresses the need for a mobile, cost-effective solution that can operate both indoors and outdoors, bridging the gap between complex deep learning models and practical "on-the-edge" deployment.

Methodology

The researchers developed a hardware-software integrated system designed for active tracking and real-time processing.

Hardware and Software Stack

The core of the system is an NVIDIA Jetson Nano, an embedded platform chosen for its ability to run deep learning workloads via its Maxwell GPU. To ensure the subject stays in the frame, an IMX415 HD camera is mounted on DS3230 digital steering gears, allowing for 360-degree dynamic tracking. The software framework utilizes MediaPipe for pose estimation, specifically calling the poseLandmarkerResult function to extract 3D coordinates of skeletal key points.

Trajectory Modeling

To analyze motion, the system focuses on the relationship between nodes (e.g., the shoulder, elbow, and wrist). The angular change ($\theta$) between these nodes is calculated over time using the dot product formula:

$$\theta = \arccos\left(\frac{\vec{AB} \cdot \vec{BC}}{|\vec{AB}| \cdot |\vec{BC}|}\right)$$

Joint angle calculation using skeletal key points

This allows the system to quantify the "correctness" of a movement by comparing the user's angular trajectory against professional reference data.

Key Insights and Applications

  • Active Tracking at the Edge: By integrating steering gears, the camera can "follow" the athlete, which is crucial for dynamic sports like tennis or ping-pong where the subject moves rapidly across a wide area.
  • Idea Transfer: The logic used to track human limbs can be adapted to track the movement of animals (such as snakes or octopuses). This suggests that the MediaPipe + Edge Hardware architecture could be a powerful tool for biologists and livestock monitoring.
  • Multimodal Integration: The system combines pose estimation with object detection (via OpenCV) to identify sports equipment like tennis balls, providing a holistic view of athletic performance.

Results and Discussion

The system successfully demonstrated real-time visualization of 3D trajectories on the Jetson Nano, providing a visual "trace" of the athlete's motion. The implementation of the angular distance formula allowed for quantitative analysis of joint movement, which serves as the primary metric for posture correction.

Real-time motion trajectory analysis during a ping-pong session

Tests in ping-pong and tennis scenarios confirmed that the MediaPipe/OpenCV stack is lightweight enough to run on the Jetson Nano’s Ubuntu Linux environment while maintaining the performance needed for real-time feedback.

Conclusion and Future Work

While the current prototype establishes a functional baseline for edge-based motion analysis, future work will focus on quantitative benchmarking (e.g., FPS and accuracy metrics) and the development of automated "coaching" algorithms. These algorithms would ideally provide natural-language feedback in real-time to guide athletes toward optimal form. Additionally, expanding the system to animal behavior analysis presents a significant opportunity for cross-disciplinary application.

Summary updated

Abstract

This paper investigates the posture trajectories in human motion using pose recognition technology based on the deep learning framework MediaPipe.By detecting key points on the human body and plotting and calculating these trajectories in the temporal dimension, we further conduct comparative analysis of these trajectories with professional sports coaches' motion guidance to assist athletes in correcting their posture.Additionally, this technology has been deployed on Jetson Nano, enabling its practical application in mobile scenarios, and providing robust tools and methods for fields such as rehabilitation therapy, sports training, and animal behavior analysis.This study offers insights into the transfer applications of posture recognition.
BibTeX
@inproceedings{zhang2024motion,
  author = {Yuhao Zhang and Mingyue Li and Jianhao Jiao},
  title = {{Motion Analysis and Transfer Applications Based on Posture Recognition}},
  booktitle = {Proceedings of International Conference on Artificial Life and Robotics},
  year = {2024},
  doi = {10.5954/icarob.2024.os4-4},
}
Figure