TL;DR
This paper rigorously evaluates the positioning performance of Vicon motion capture systems, a widely used technology in fields ranging from biomechanics to robotics. By developing a novel ground truth setup that doesn't rely on other motion capture systems, the authors provide a more accurate assessment of Vicon's static and dynamic accuracy, revealing that performance is significantly influenced by marker size and sampling rate relative to object speed.
What Problem Does This Paper Address?
Motion capture systems, particularly Vicon, are ubiquitous for tracking objects and humans in 3D space. While their applications are diverse, from clinical gait analysis to guiding autonomous robots like UAVs, there's a surprising lack of detailed studies specifically evaluating the positioning performance of these systems. Existing research often relies on indirect measures like inter-marker distances or uses other motion capture systems as ground truth, which can propagate errors. This paper aims to fill this gap by providing a comprehensive, marker-wise evaluation of Vicon's static and dynamic positioning accuracy using a novel, independent ground truth setup. This is crucial for applications where precise positioning is critical, such as robotics and augmented reality.
Methods
The authors employ a two-pronged approach to evaluate Vicon's performance:
- Static Experiments: They utilize a 4-axis robot with a known 10 µm resolution to move a probe within a 2x1.5x1 m³ workspace. A Vicon marker is attached to the probe, and its position is recorded in both the robot's and Vicon's reference frames. A sophisticated least-squares method (based on Umeyama's algorithm) is used to estimate the rigid transform between these frames, allowing for precise quantification of Vicon's static accuracy and repeatability.
- Dynamic Experiments: To assess performance under motion, they designed a rotating blade setup. This system includes an electric motor, a gearbox, and a quadrature encoder providing a high-resolution (75 kHz) ground truth angular position. Four markers are placed on the blade. The challenge here is to establish a transformation between the Vicon frame and the ground truth angular frame. They propose two methods for this extrinsic calibration:
- Uyemana Method: A direct least-squares estimation of the transformation parameters.
- Plane + Circle: This method leverages the fact that markers on a rotating blade trace a plane. It involves estimating the plane equation, fitting a circle to the projected points, and then estimating an angular offset.
The Euclidean distance is used as the primary metric for evaluating positioning performance in both static and dynamic scenarios.
Datasets
The paper does not explicitly mention training datasets as it focuses on evaluating an existing system. However, it introduces a novel experimental setup for dynamic evaluation.
- Static Dataset: Generated using a robot arm moving a probe with a Vicon marker in a controlled 2x1.5x1 m³ volume.
- Dynamic Datasets: Eight datasets were collected using the rotating blade setup. These datasets vary in motor speed (from 100 to 1711 rpm) and Vicon sampling rate (100 Hz and 200 Hz). Dataset 8 was problematic due to high speeds causing markers to detach.
Key Insights and Ideas
- Ground Truth Matters: The paper highlights the importance of an independent, high-accuracy ground truth system for evaluating motion capture performance, moving beyond reliance on inter-marker distances or other motion capture systems.
- Dynamic Performance is Speed-Dependent: Contrary to some assumptions, the study found that Vicon's error decreases as the speed of the object increases, up to a certain point. This is attributed to the trade-off between marker size and sampling rate, where at low speeds, marker movement might fall within the camera's segmentation noise.
- Marker Size and Sampling Rate are Crucial: The authors emphasize that optimal performance requires careful tuning of marker size and Vicon sampling rate relative to the expected speeds in the application.
- Inner vs. Outer Markers: Smaller diameter markers (like marker B on the blade) generally exhibit better performance than larger diameter markers (A and D), potentially due to better correction of distortions in the center of the camera's field of view.
- Static Performance is Excellent: Vicon systems demonstrate very high precision and accuracy in static scenarios, with mean absolute errors around 0.15 mm and low variability.
Key Results
- Static Performance:
- Mean Absolute Error (MAE): 0.153 mm
- Root Mean Squared Error (RMSE): 0.154 mm
- Positioning variability (standard deviation): < 0.025 mm
- Dynamic Performance:
- Errors can be reduced by up to 40% for the same object moving at higher speeds.
- The system error is generally less than 2 mm for speeds encountered in real applications.
- Optimal positioning performance can reach 0.3 mm for dynamic studies.
- For speeds greater than 3 m/s, sub-millimeter positioning error is achieved.
- For speeds less than 1 m/s, errors are still considered fair, below 2 mm.
Limitations and Future Work
- Limitations:
- Dataset 8 in the dynamic experiments was compromised due to markers detaching at high speeds.
- The study primarily focuses on circular trajectories; other motion patterns might yield different results.
- The paper acknowledges that marker size and sampling rate are critical and need to be tuned, implying that suboptimal choices can lead to degraded performance.
- Future Work:
- Perform linear motion experiments at speeds comparable to the circular trajectories.
- Mimic more complex human or animal motions using a Kuka robot.
- Evaluate motions similar to UAV acrobatic maneuvers at higher speeds and with less constrained trajectories.
- Further investigate the influence of marker size and sampling rate on performance.
Summary updated