TL;DR
This paper introduces a novel method to improve the accuracy of human joint tracking from low-cost motion capture systems like Microsoft Kinect, especially in challenging scenarios with occlusions. By classifying joint data into "noise" and "erroneous" categories and applying tailored filtering techniques, the method significantly enhances tracking accuracy compared to existing approaches.
What Problem Does This Paper Address?
Low-cost motion capture systems, such as Microsoft Kinect, offer a promising avenue for applications ranging from human-robot interaction and rehabilitation to clinical assessments. These systems track 3D skeleton joints on the human body, providing valuable motion data. However, the raw data from Kinect is often plagued by jitter, noise, and estimation errors, which are exacerbated when parts of the body are temporarily occluded or move out of the sensor's field of view. This unreliability can severely impact the performance and accuracy of applications relying on this data. While existing methods like moving average filters and traditional Kalman filters can smooth out some noise, they struggle to effectively handle significant errors caused by occlusions, often propagating these errors through the system. This paper aims to address this gap by developing a more robust method that can differentiate between noise and actual erroneous data and process them accordingly.
Methods
The proposed method combines data classification with a refined Kalman filtering approach.
- Reliability Measurement: A "vibration degree" index is introduced to quantify the reliability of each joint's position across consecutive frames. This index is based on the angle between displacement vectors of a joint. A high vibration degree suggests potential issues.
- Data Classification: Based on the reliability index, joint data is classified into two categories:
- Noise Data: Joints with high reliability (above a threshold of 0.70) are considered noise and are smoothed using a standard Kalman filter.
- Erroneous Data: Joints with low reliability (below 0.70) are identified as erroneous, likely due to occlusion or significant estimation errors.
- Handling Noise Data: For noise data, a Kalman filter is employed. The state vector includes joint position and velocity, while the measurement vector incorporates joint position and the sound source angle (to leverage additional Kinect sensor data). An extended Kalman filter is used to handle the non-linear measurement model.
- Handling Erroneous Data: For erroneous data, a more sophisticated approach is used. First, a Kalman filter estimates the motion trend. Then, human body constraints (specifically, the constant length of the limb connecting the erroneous joint to its parent joint) are used to optimize the estimated position. This involves finding a point on a sphere (defined by the parent joint and limb length) that is closest to the Kalman filter's prediction, effectively correcting the erroneous joint's position based on anatomical plausibility.
Datasets
The paper does not explicitly mention training datasets. For evaluation, they collected data from five subjects performing a specific motion.
- Experimental Data: Data was collected from five subjects performing a quarter-circular trajectory with their wrist. To simulate occlusion, an obstruction was introduced into the joint's trajectory. Each subject performed five experiments, with each experiment collecting 120 frames of data. The ground truth trajectory was established by having subjects move along a quarter-circular path while holding a tape measure attached to their wrist, with the tape measure's fixed point defining the trajectory.
Key Insights and Ideas
- The importance of data classification: The core insight is that treating all noisy or inaccurate data the same is suboptimal. Differentiating between minor noise and significant errors (like those from occlusion) allows for more targeted and effective correction.
- Leveraging human body constraints: For erroneous data, simply smoothing might not be enough. Incorporating anatomical constraints (like fixed limb lengths) provides a powerful mechanism to correct highly inaccurate joint positions in a physically plausible way.
- Combined approach is key: The success of the method lies in the synergy between reliability assessment, classification, and specialized filtering for each data type. This is more effective than a single-stage filtering approach.
Key Results
The proposed method demonstrated significant improvements in tracking accuracy compared to the original Kinect data, a moving mean filter, and a traditional Kalman filter.
- Accuracy Improvement: The method achieved an accuracy improvement of 58.7% compared to the original Kinect data, 47.5% compared to the moving mean filter, and 22.5% compared to the traditional Kalman filter.
- Reduced Error: Table 2 shows that the mean error for the proposed method was 0.031 m, significantly lower than Kinect (0.075 m), Moving Mean Filter (0.059 m), and Kalman Filter (0.040 m).
- Qualitative Improvement: Figure 6 visually demonstrates that the proposed method's trajectory closely follows the ground truth, even through an occlusion event, while other methods show larger deviations.
Limitations and Future Work
The authors acknowledge several limitations and suggest future research directions:
- Limitations:
- The reliability classification currently only considers the vibration between frames. The relationship between joint coordinates within the same frame is not fully utilized.
- The reliability threshold (0.70) and other parameters (like $\theta_{min}$, $\theta_{max}$) were empirically determined and may require expert consultation for more robust settings.
- The evaluation was primarily conducted on a specific quarter-circular wrist trajectory.
- Future Work:
- Enrich the skeletal joint data classification algorithm by incorporating inter-frame joint relationships.
- Consult with experts in physiology, rehabilitation, or neuroscience to determine more scientifically grounded thresholds.
- Verify the method's performance on more complex human motions and potentially explore more sophisticated ground truth acquisition methods (which may require more advanced equipment).
Summary updated