Optimization-Based Online Initialization and Calibration of Monocular Visual-Inertial Odometry Considering Spatial-Temporal Constraints

Weibo Huang, Weiwei Wan, Hong Liu

TL;DR

This paper introduces a novel method for simultaneously initializing and calibrating monocular Visual-Inertial Odometry (VIO) systems. It addresses the critical need for accurate spatial and temporal sensor alignment without requiring prior knowledge, outperforming existing methods in robustness and accuracy, especially under challenging conditions.

What Problem Does This Paper Address?

Visual-Inertial Odometry (VIO) systems fuse camera and Inertial Measurement Unit (IMU) data to estimate a robot's motion and build a 3D map. For monocular VIO, achieving accurate results hinges on two crucial aspects: initializing the system's state (like scale, velocity, and biases) and calibrating the spatial (how the camera and IMU are positioned relative to each other) and temporal (the time difference between their measurements) parameters. Many existing VIO systems rely on filter-based approaches, but optimization-based VIOs, known for their higher accuracy, often lack robust online methods for this crucial initialization and calibration phase. This gap means that setting up these systems can be cumbersome, requiring manual calibration or prior knowledge, which is impractical for many real-world applications.

Methods

The proposed method tackles this challenge in three main steps:

  1. Time Offset Modeling and Sensor Motion Interpolation: It first models the time offset between camera and IMU data. Then, it employs two short-term motion interpolation algorithms. These algorithms assume the sensors move with constant angular and linear velocities over short intervals to align asynchronous camera and IMU measurements. Crucially, the camera motion interpolation accounts for the unknown metric scale, allowing for the formulation of transformations between camera and IMU poses as a function of the temporal parameter.
  2. Incremental Estimation: The aligned and interpolated data are fed into an incremental estimator. This estimator iteratively refines the initial states (metric-scale, velocity, gravity, IMU biases) and calibrates the spatial-temporal parameters. It works in a three-step process: first estimating rotation, time offset, and gyroscope bias; then estimating scale, gravity, and spatial translation; and finally refining these by considering accelerometer bias.
  3. Bundle Adjustment: A tightly-coupled nonlinear optimization (bundle adjustment) is then applied. This step further improves the accuracy of the estimated states and parameters by minimizing both IMU preintegration errors and feature reprojection errors, effectively optimizing the IMU states, spatial-temporal parameters, and the reconstructed map.

Datasets

The paper evaluates its method using:

  • Synthetic Datasets: Used to analyze the robustness of the incremental estimator against various IMU sensor noises and time offsets.
  • EuRoC Dataset: A widely-used real-world dataset containing sequences from both a machine hall (MH) and a room (V1, V2) with varying difficulty levels. This dataset provides ground-truth for IMU biases, spatial parameters, and trajectories.
  • Real Sensor Experiment: Utilized data from an Intel RealSense D435i depth camera, which includes an accelerometer, gyroscope, and rolling shutter RGB camera. This experiment specifically investigated the impact of varying camera exposure times on time offset calibration.

Key Insights and Ideas

  • Bridging the Gap for Optimization-Based VIO: The paper successfully addresses a significant gap by providing an online spatial-temporal calibration method specifically for optimization-based VIO, which are generally more accurate than filter-based methods.
  • No Prior Knowledge Required: A key strength is that the method does not need any pre-existing information about the spatial or temporal configuration of the sensors, making it highly practical for "power-on-and-go" scenarios.
  • Robustness to Time Offsets: The proposed method demonstrates remarkable robustness to varying time offsets, maintaining high accuracy even when the offset is significant, a challenge for many existing approaches.
  • Coarse-to-Fine Calibration: The incremental estimator provides a coarse estimate, which is then refined by the bundle adjustment, leading to a more accurate and complete calibration.

Key Results

  • Accurate Calibration: The method accurately estimates initial states and spatial-temporal parameters. Experiments show that calibrated parameters are close to ground-truth values.
  • Outperforms State-of-the-Art: The proposed method consistently outperforms existing methods like VINS-Mono and Feng et al.'s approach in terms of trajectory accuracy (RMSE) across various time offsets and datasets. For instance, it achieved an average trajectory RMSE of 0.035 m on the MH_03 sequence, significantly lower than other methods.
  • Robustness to Noise: Simulation results indicate that the method is robust to various IMU sensor noises (measurement noise, biases, and bias diffusion), with calibration errors remaining within acceptable limits.
  • Efficient Convergence: The incremental estimator converges to stable parameter estimates within a few seconds, making it suitable for real-time applications.
  • Real-World Validation: Experiments with the RealSense D435i camera confirmed the method's ability to accurately calibrate spatial-temporal parameters and its robustness to different camera exposure times.

Limitations and Future Work

  • Monocular VIO Focus: The current work is specifically designed for monocular VIO.
  • Future Extension: The authors propose extending the method to other types of odometers, such as stereo/dual visual-inertial odometry, Lidar-visual-inertial odometry, encoder-visual-inertial odometry, and GPS-visual-inertial odometry.

Summary updated

Abstract

The online system state initialization and simultaneous spatial-temporal calibration are critical for monocular Visual-Inertial Odometry (VIO) since these parameters are either not well provided or even unknown. Although impressive performance has been achieved, most of the existing methods are designed for filter-based VIOs. For the optimization-based VIOs, there is not much online spatial-temporal calibration method in the literature. In this paper, we propose an optimization-based online initialization and spatial-temporal calibration method for VIO. The method does not need any prior knowledge about spatial and temporal configurations. It estimates the initial states of metric-scale, velocity, gravity, Inertial Measurement Unit (IMU) biases, and calibrates the coordinate transformation and time offsets between the camera and IMU sensors. The work routine of the method is as follows. First, it uses a time offset model and two short-term motion interpolation algorithms to align and interpolate the camera and IMU measurement data. Then, the aligned and interpolated results are sent to an incremental estimator to estimate the initial states and the spatial–temporal parameters. After that, a bundle adjustment is additionally included to improve the accuracy of the estimated results. Experiments using both synthetic and public datasets are performed to examine the performance of the proposed method. The results show that both the initial states and the spatial-temporal parameters can be well estimated. The method outperforms other contemporary methods used for comparison.
BibTeX
@inproceedings{huang2021optimization,
  author = {Weibo Huang and Weiwei Wan and Hong Liu},
  title = {{Optimization-Based Online Initialization and Calibration of Monocular Visual-Inertial Odometry Considering Spatial-Temporal Constraints}},
  booktitle = {Sensors},
  year = {2021},
  url = {https://openreview.net/forum?id=rMcl3hBJgE},
}
Figure