Online Multi-IMU Calibration Using Visual-Inertial Odometry

Jacob Hartzer, Srikanth Saripalli

TL;DR

This paper introduces a novel filter framework that allows for the online calibration of multiple, unsynchronized Inertial Measurement Units (IMUs) within a Visual-Inertial Odometry (VIO) system. This approach improves the accuracy and robustness of robot navigation by leveraging all available IMU data, even when sensors have varying quality or drift over time.

What Problem Does This Paper Address?

Modern robots rely heavily on sensors like IMUs for accurate navigation, especially in challenging environments. IMUs provide high-rate estimates of motion, crucial for stabilizing VIO systems. However, the accuracy of these estimates is highly sensitive to the IMU's intrinsic (e.g., bias) and extrinsic (e.g., position and orientation relative to the robot) calibration parameters. These parameters can drift over time, and traditional calibration methods often require specific, controlled environments and offline processing. Furthermore, using multiple IMUs to improve robustness can be inefficient if not all data is optimally utilized. This paper addresses the challenge of accurately and robustly calibrating multiple, potentially unsynchronized IMUs in real-time, directly within a VIO system, without relying on rigid geometric constraints.

Methods

The core of this work is a centralized Extended Kalman Filter (EKF) framework that extends the Multi-State Constraint Kalman Filter (MSCKF) commonly used in VIO. The novel aspects include:

  • Multi-IMU State Extension: The filter's state vector is augmented to include not only the robot's pose, velocity, and acceleration but also the intrinsic and extrinsic calibration parameters for each individual IMU.
  • Online Calibration: The filter is designed to estimate these calibration parameters online, meaning they are updated continuously as new sensor data arrives. This is crucial for handling parameter drift.
  • Flexible Sensor Integration: The framework can accommodate any number of IMUs, regardless of their update rates or synchronization. It doesn't assume perfect synchronization between IMUs, which is a significant advantage over many existing methods.
  • Leveraging Visual Data: While the focus is on IMU calibration, the framework is designed to be integrated with exteroceptive measurements (like camera data in an MSCKF) to ensure the observability of all calibration states. This is because IMU measurements alone are insufficient to fully estimate all calibration parameters.

Datasets

The paper does not explicitly mention specific public datasets used for training or evaluation. Instead, it relies on:

  • Monte Carlo Simulations: A custom simulation environment was developed to generate random 6-DOF trajectories and introduce various errors into IMU calibration parameters. This allowed for extensive testing of the filter's stability and robustness under controlled conditions.
  • Experimental Data: The authors conducted experiments using a real-world setup with multiple IMUs (VectorNav VN-300, VectorNav VN-100, and FDI Systems DETA-10) and a camera. Ground truth was obtained using a GPS-aided INS solution from the VN-300.

Key Insights and Ideas

  • Online Calibration is Key: The ability to estimate calibration parameters in real-time is critical for maintaining navigation accuracy, especially when IMU biases drift over time.
  • More IMUs, Better Results (with caveats): Using multiple IMUs generally leads to improved VIO accuracy. Interestingly, the simulations suggest that adding two independent, lower-quality IMUs can be as beneficial as adding a single high-quality, high-rate IMU, offering a trade-off for systems with computational limitations.
  • Observability Requires Exteroception: While the filter can estimate calibration parameters, full observability of all intrinsic and extrinsic parameters for every IMU requires exteroceptive measurements (like camera data). IMUs alone cannot fully resolve all calibration ambiguities.
  • Flexibility is a Major Advantage: The framework's ability to handle unsynchronized sensors and varying quality is a significant step beyond methods that require strict synchronization or assume identical sensor performance.

Key Results

  • Improved VIO Accuracy: Simulations showed a general trend of increasing VIO accuracy as the number of IMUs increased and their individual measurement errors decreased. The calibration filter demonstrated significant reductions in relative error compared to non-calibration methods.
  • Accurate Calibration Estimates: Both simulations and experiments showed that the proposed filter produces consistent and accurate estimates of initial calibration errors.
  • Comparable to State-of-the-Art: Experimental results showed that the filter produces similar intrinsic and extrinsic calibration parameters compared to the state-of-the-art offline optimizer Kalibr.
  • Robustness: The filter demonstrated stability and robustness against changes in calibration parameters and even measurement stream drop-outs.

Limitations and Future Work

  • Observability Dependency: As noted, the filter's ability to fully estimate all calibration parameters relies on the availability of exteroceptive measurements. Without them, some states remain unobservable.
  • Computational Cost: While the framework is designed to be efficient, integrating multiple IMUs and estimating their calibration parameters will inherently increase computational load compared to a single IMU system.
  • Future Work: The authors suggest that this framework can be further extended to incorporate other sensor modalities and to handle more complex sensor failure scenarios. The open-source nature of the code also invites further development and application by the research community.

Summary updated

Abstract

This work presents a centralized multi-IMU filter framework with online intrinsic and extrinsic calibration for unsynchronized inertial measurement units that is robust against changes in calibration parameters. The novel EKF-based method estimates the positional and rotational offsets of the system of sensors as well as their intrinsic biases without the use of rigid body geometric constraints. Additionally, the filter is flexible in the total number of sensors used while leveraging the commonly used MSCKF framework for camera measurements. The filter framework has been validated using Monte Carlo simulation as well as experimentally. In both simulations and experiments, using multiple IMU measurement streams within the proposed filter framework outperforms the use of a single IMU in a filter prediction step while also producing consistent and accurate estimates of initial calibration errors. Compared to current state-of-the-art optimizers, the filter produces similar intrinsic and extrinsic calibration parameters for each sensor. Finally, an open source repository has been provided at https://github.com/unmannedlab/ekf-cal containing both the online estimator and the simulation used for testing and evaluation.
BibTeX
@inproceedings{hartzer2023online,
  author = {Jacob Hartzer and Srikanth Saripalli},
  title = {{Online Multi-IMU Calibration Using Visual-Inertial Odometry}},
  booktitle = {Sdf-mfi},
  year = {2023},
  doi = {10.1109/sdf-mfi59545.2023.10361310},
}
Figure