Evaluation of the Pose Tracking Performance of the Azure Kinect and Kinect v2 for Gait Analysis in Comparison with a Gold Standard: A Pilot Study

Justin Amadeus Albert, Victor Owolabi, Arnd Gebel, Clemens Markus Brahms, Urs Granacher, Bert Arnrich

TL;DR

This study compares the Azure Kinect and Kinect v2 for gait analysis, a crucial tool for detecting neurological diseases and fall risk. The Azure Kinect demonstrates significantly higher accuracy in spatial gait parameters compared to its predecessor, the Kinect v2, while temporal parameters show comparable performance. This suggests the Azure Kinect is a promising, more accurate, and cost-effective alternative for clinical gait analysis.

What Problem Does This Paper Address?

Gait analysis is vital for understanding health, particularly in detecting neurological conditions like Parkinson's disease and assessing fall risk in the elderly. Traditionally, this has relied on expensive and complex multi-camera motion capture systems. While the Microsoft Kinect has offered a more accessible solution, its accuracy has been a concern. This paper addresses the need to evaluate the performance of the latest Azure Kinect, with its advanced hardware and deep learning-based tracking, against its predecessor (Kinect v2) and a gold-standard Vicon system for gait analysis. The goal is to determine if the Azure Kinect can provide sufficiently accurate data for clinical applications, potentially democratizing access to gait analysis.

Methods

The researchers employed a rigorous comparison methodology:

  • Gold Standard: A Vicon motion capture system with a 39-marker Plug-in Gait model served as the ground truth.
  • Kinect Setup: Both the Kinect v2 and Azure Kinect cameras were positioned in front of a treadmill.
  • Participants: Five young, healthy individuals walked on the treadmill at three different speeds (3.0, 3.9, and 4.7 km/h).
  • Calibration: A novel, easy-to-administer camera calibration method was developed to spatially align the 3D skeleton data from the Kinect cameras and the Vicon system. This involved using a checkerboard pattern and retro-reflective markers.
  • Data Processing: Data from all systems were synchronized and filtered. Kinect data were upsampled to 100 Hz to match the Vicon system's sampling frequency.
  • Evaluation Metrics:
    • Spatial Agreement: 3D Euclidean distances between corresponding joints of the Kinect skeletons and the Vicon markers were calculated.
  • Spatio-Temporal Gait Parameters: Key parameters like step length, step width, step time, and stride time were calculated using an established algorithm.
  • Statistical Analysis: Paired Wilcoxon tests and Pearson correlation coefficients were used to assess statistical significance and agreement. Absolute error, relative error, and Root Mean Squared Error (RMSE) were calculated for gait parameters.

Datasets

The study did not use pre-existing datasets for training. Instead, it collected new data from five healthy participants performing treadmill walking. This dataset was used for evaluating the performance of the Azure Kinect and Kinect v2 against the Vicon gold standard.

Key Insights and Ideas

  • Azure Kinect's Foot Tracking Superiority: The Azure Kinect demonstrated significantly higher accuracy in tracking foot markers, which is crucial for accurate step length and width calculations. This is a key differentiator from the Kinect v2.
  • Upper Body vs. Lower Extremities: While Azure Kinect excelled in foot tracking, the Kinect v2 showed slightly better performance for upper body joints. This suggests a trade-off or specific strengths for each sensor depending on the body part of interest.
  • Deep Learning Advantage: The paper highlights the potential of the Azure Kinect's deep learning-based body tracking SDK, which likely contributes to its improved accuracy.
  • Calibration is Key: The development of a straightforward calibration method is presented as a valuable contribution, enabling accurate spatial alignment between different motion capture systems.
  • Temporal vs. Spatial Accuracy: The study clearly distinguishes between spatial and temporal gait parameters, finding significant improvements in spatial accuracy with the Azure Kinect, but no significant differences in temporal accuracy.

Key Results

  • Spatial Agreement: The Azure Kinect showed significantly higher accuracy for foot markers compared to Kinect v2. For example, the Pearson correlation for the vertical direction of foot markers was moderate to good for Azure Kinect (0.72-0.71) versus poor for Kinect v2 (0.11-0.01).
  • Spatio-Temporal Gait Parameters:
    • Step Length & Width: Azure Kinect consistently outperformed Kinect v2 across all treadmill speeds, with smaller absolute errors, relative errors, and RMSE values. For instance, the RMSE for step length was 0.04 m for Azure Kinect vs. 0.08 m for Kinect v2.
  • Step Time & Stride Time: Both Kinect generations performed similarly, with no significant differences in error values or RMSE.
  • Gait Event Timing: Both Kinect sensors showed comparable accuracy in detecting heel strike and toe-off events, with mean errors within an acceptable range.

Limitations and Future Work

  • Pilot Study & Sample Size: This was a pilot study with a small sample size (five young, healthy participants). The results are specific to this demographic and their typical walking speeds.
  • Unpathological Gait: The study focused on participants with normal gait patterns. Further validation is needed for individuals with gait abnormalities (e.g., Parkinson's, stroke).
  • Future Work: The authors suggest validating the Azure Kinect with diverse age groups and individuals with gait abnormalities. They also propose investigating its use for body weight exercises and overground walking. Further research should also consider the impact of future SDK updates on tracking quality.

Summary updated

BibTeX
@inproceedings{albert2020evaluation,
  author = {Justin Amadeus Albert and Victor Owolabi and Arnd Gebel and Clemens Markus Brahms and Urs Granacher and Bert Arnrich},
  title = {{Evaluation of the Pose Tracking Performance of the Azure Kinect and Kinect v2 for Gait Analysis in Comparison with a Gold Standard: A Pilot Study}},
  booktitle = {Sensors},
  year = {2020},
  url = {https://openreview.net/forum?id=fWaN7aYMOxvh},
}
Figure