Hand-Writing Motion Tracking with Vision-Inertial Sensor Fusion: Calibration and Error Correction

Shengli Zhou, Fei Fei, Guanglie Zhang, Yunhui Liu, Wen J. LiSensors 2014 · DOI

TL;DR

This paper tackles the challenge of accurate 3D motion tracking by fusing data from vision and inertial sensors. They introduce a novel approach to calibrate these sensors and model their inherent errors, leading to significantly improved tracking accuracy compared to existing methods.

What Problem Does This Paper Address?

Real-time 3D motion tracking is crucial for applications like human-computer interaction and augmented reality. However, relying on a single sensor type presents limitations: vision sensors offer good accuracy but struggle with fast movements and can have slow update rates, while inertial sensors (like accelerometers and gyroscopes) are good for fast motion but suffer from rapid error accumulation (drift). This paper aims to overcome these limitations by fusing data from both vision and inertial sensors, specifically addressing the challenges of sensor calibration and accurately modeling the noise and biases inherent in inertial sensors.

Methods

The authors propose a multi-stage approach:

  • Sensor Calibration: They developed algorithms for calibrating the relative rotations between different coordinate frames (inertial sensor, camera, and world). Notably, they claim this can be achieved using just one reference image.
  • Deterministic Error Calibration: For inertial sensors, they employ a Gauss-Newton method to iteratively refine calibration parameters, addressing biases, scale factors, and non-orthogonalities. This method is chosen for its efficiency and ease of implementation.
  • Stochastic Error Identification and Modeling: They utilize Allan Variance analysis to identify and characterize stochastic errors in inertial sensors, such as angle/velocity random walk and bias instability. These identified errors are then modeled using a first-order Gauss-Markov process for bias instability and white noise for random walk.
  • Sensor Fusion: The calibrated and error-modeled inertial sensor data is fused with vision sensor data using an extended Kalman filter. This fusion aims to leverage the strengths of each sensor while mitigating their weaknesses.

Datasets

Not discussed in the paper. The experiments involved writing English characters ("cityu") on a whiteboard using a custom-built system.

Key Insights and Ideas

  • Single Reference Image for Calibration: The claim that relative rotations can be calibrated with just one reference image is a significant simplification compared to methods requiring multiple views.
  • Allan Variance for Inertial Sensor Characterization: Using Allan Variance is a standard but effective method for understanding the noise characteristics of inertial sensors, allowing for more accurate modeling.
  • Gauss-Markov for Bias Instability: Modeling bias instability as a Gauss-Markov process is a well-established technique that provides a good balance between accuracy and computational complexity.
  • Fusion for Enhanced Accuracy: The core idea that fusing well-calibrated and error-modeled inertial data with vision data can significantly improve ego-motion tracking accuracy is validated by their results.

Key Results

  • The proposed error correction model significantly enhances ego-motion tracking accuracy compared to conventional sensor fusion models.
  • Specifically, the RMSE of reconstructed hand-writing trajectories was reduced by 6.75 mm over a 30 cm by 15 cm writing area.
  • The total RMSE in three directions was reduced by 20.17 mm.
  • The algorithm demonstrated the ability to recover more detailed information of real movement, as evidenced by trajectory comparisons.
  • Dynamic Time Warping (DTW) was successfully applied for temporal information retrieval.

Limitations and Future Work

  • Radial Distortion: The paper acknowledges that the radial distortion of the web-based camera was not fully resolved, leading to a "stretching" effect in the x-direction (radial direction of the camera), which can deteriorate fusion results.
  • Image Blurring: Fast movements can cause image blurring, leading to the loss of vision sensor data. While inertial sensors can provide pose estimates during these periods, their accuracy deteriorates quickly if vision data is frequently lost.
  • Future Work: The authors suggest resolving camera radial distortion and developing more robust methods to handle image blurring during fast movements. They also envision realizing a mobile motion tracking and recognition system based on their fusion approach.

Summary updated

Abstract

The purpose of this study was to improve the accuracy of real-time ego-motion tracking through inertial sensor and vision sensor fusion. Due to low sampling rates supported by web-based vision sensor and accumulation of errors in inertial sensors, ego-motion tracking with vision sensors is commonly afflicted by slow updating rates, while motion tracking with inertial sensor suffers from rapid deterioration in accuracy with time. This paper starts with a discussion of developed algorithms for calibrating two relative rotations of the system using only one reference image. Next, stochastic noises associated with the inertial sensor are identified using Allan Variance analysis, and modeled according to their characteristics. Finally, the proposed models are incorporated into an extended Kalman filter for inertial sensor and vision sensor fusion. Compared with results from conventional sensor fusion models, we have shown that ego-motion tracking can be greatly enhanced using the proposed error correction model.
BibTeX
@inproceedings{zhou2014hand,
  author = {Shengli Zhou and Fei Fei and Guanglie Zhang and Yunhui Liu and Wen J. Li},
  title = {{Hand-Writing Motion Tracking with Vision-Inertial Sensor Fusion: Calibration and Error Correction}},
  booktitle = {Sensors},
  year = {2014},
  doi = {10.3390/s140915641},
}
Figure