Visual Gait Analysis Based on UE4

Ruzhang Liu, Luyin Liu, Guochao Ma, Shanshan Feng, Yuanhui Mu, Dexi Meng, Shuying Wang, Enlin CaiSensors 2023 · DOI

TL;DR

This paper presents a novel 3D human body animation model built in Unreal Engine 4 (UE4) that visualizes gait analysis data captured by inertial sensors in real-time. By integrating inertial sensor data with a sophisticated UE4 model and employing error correction mechanisms, the system offers a more intuitive and accurate representation of human motion for medical and sports applications.

What Problem Does This Paper Address?

Traditional gait analysis often relies on subjective, qualitative observations or complex, less intuitive quantitative methods. While 3D gait analysis has advanced, there's a gap in effectively visualizing this data in real-time, especially for understanding subtle gait deviations that might be missed by the naked eye. Existing methods often focus on specific limb movements or lack a comprehensive, visually engaging human model. This paper aims to bridge this gap by creating a dynamic, interactive 3D human model that can accurately display and analyze gait in real-time, aiding in diagnosis and understanding of movement disorders.

Methods

The core of this work lies in the integration of inertial sensor data with a 3D human model developed within the Unreal Engine 4 (UE4) platform. The authors utilized UE4's blueprint visual scripting and C++ programming to construct a hierarchical skeleton model of the human body. This model comprises 12 key body parts, each with its own independent coordinate system for analyzing motion.

Key methodological aspects include:

  • 3D Pose Model Creation: A detailed 3D human model was built in UE4, featuring a skeletal structure that allows for animation.
  • Inertial Sensor Integration: The system is designed to receive motion data from inertial sensors (like IMUs). This data is transmitted via a virtual serial port.
  • Data Processing: Raw motion data is processed using a Kalman filter and a complementary filter attitude algorithm to reduce errors and improve accuracy.
  • Quaternion to Euler Angle Conversion: The processed data, often in quaternion format from sensors, is converted to Euler angles for application to the model's joints.
  • Ergonomic Linkage and Error Correction: A crucial aspect is the implementation of an "ergonomic linkage" where all joints are interrelated. This prevents unrealistic movements and allows for automatic calibration and correction of errors from inertial sensors. For instance, if one joint's data suggests it's separating from the body, the system corrects the data to maintain structural integrity.
  • Real-time Visualization: The UE4 engine's capabilities are leveraged to display the 3D human model's posture and motion in real-time, providing a dynamic visualization of gait.
  • Custom Blueprints: Specific blueprints were developed for setting up virtual serial ports, receiving and processing joint data (e.g., lumbar joint), and enabling mouse-based control for debugging and interaction.

Datasets

Not discussed in the paper. The paper focuses on the system's design and real-time data processing rather than training on a specific dataset. The input data is derived from inertial sensors during motion capture.

Key Insights and Ideas

  • UE4 as a Powerful Visualization Tool for Biomechanics: The paper demonstrates that UE4, primarily known for game development, is a highly capable platform for creating realistic and interactive 3D models for scientific applications like gait analysis.
  • The Importance of Ergonomic Linkage: The concept of interlinking all joints in the 3D model is critical. It not only prevents physically impossible movements but also allows for error compensation from sensor data, significantly improving the reliability of the visualized motion.
  • Real-time Error Correction Enhances Accuracy: By actively correcting for potential errors in inertial sensor measurements, the system ensures that the visualized motion remains consistent with human biomechanics, making the analysis more trustworthy.
  • Bridging the Gap Between Raw Data and Visual Understanding: The system effectively translates raw sensor data into a visually intuitive animation, making complex gait analysis more accessible to clinicians and researchers.

Key Results

The paper demonstrates that the developed 3D pose model can:

  • Vividly Display Gait Changes: The model can accurately represent changes in gait, including angles and displacements of 12 key body parts (legs, arms, waist, head).
  • Real-time Motion Visualization: It successfully displays human body posture in real-time when combined with inertial sensor motion capture.
  • Error Correction: The system can correct motion data in real-time, preventing unrealistic joint separations and improving accuracy. For example, it automatically corrects joint angles that exceed physical limits (e.g., setting a knee angle to 90° if an input of 200° is detected).
  • Functional Model: Experiments confirmed that each joint of the model can complete action displays according to motion data, and the whole model experiment showed a linked and reliable data representation.

Limitations and Future Work

  • Limitations:
    • The current system is primarily used for gait measurement and analysis of low-speed main joints.
  • Details such as hand joints and toes have not been measured or fully integrated.
  • The behavior of the whole model at high speeds and the range of motion for all joints require further verification.
  • Future Work:
    • Expanding the system to include detailed measurements of smaller joints like hands and toes.
  • Investigating the model's performance and accuracy at higher speeds of human motion.
  • Further validation of the range of motion for all joints.
  • The authors suggest the system has great application prospects in medical rehabilitation and motion analysis, implying future work will focus on these applications.

Summary updated

Abstract

With the development of artificial intelligence technology, virtual reality technology has been widely used in the medical and entertainment fields, as well as other fields. This study is supported by the 3D modeling platform in UE4 platform technology and designs a 3D pose model based on inertial sensors through blueprint language and C++ programming. It can vividly display changes in gait, as well as changes in angles and displacements of 12 parts such as the big and small legs and arms. It can be used to combine with the module of capturing motion which is based on inertial sensors to display the 3D posture of the human body in real-time and analyze the motion data. Each part of the model contains an independent coordinate system, which can analyze the angle and displacement changes of any part of the model. All joints of the model are interrelated, the motion data can be automatically calibrated and corrected, and errors measured by an inertial sensor can be compensated, so that each joint of the model will not separate from the whole model and there will not occur actions that against the human body's structures, improving the accuracy of the data. The 3D pose model designed in this study can correct motion data in real time and display the human body's motion posture, which has great application prospects in the field of gait analysis.
BibTeX
@inproceedings{liu2023visual,
  author = {Ruzhang Liu and Luyin Liu and Guochao Ma and Shanshan Feng and Yuanhui Mu and Dexi Meng and Shuying Wang and Enlin Cai},
  title = {{Visual Gait Analysis Based on UE4}},
  booktitle = {Sensors},
  year = {2023},
  doi = {10.3390/s23125463},
}
Figure