Visual-inertial self-calibration on informative motion segments

Thomas Schneider, Mingyang Li, Michael Burri, Juan I. Nieto, Roland Siegwart, Igor Gilitschenski

Abstract

Environmental conditions and external effects, such as shocks, have a significant impact on the calibration parameters of visual-inertial sensor systems. Thus long-term operation of these systems cannot fully rely on factory calibration. Since the observability of certain parameters is highly dependent on the motion of the device, using short data segments at device initialization may yield poor results. When such systems are additionally subject to energy constraints, it is also infeasible to use full-batch approaches on a big dataset and careful selection of the data is of high importance. In this paper, we present a novel approach for resource efficient self-calibration of visual-inertial sensor systems. This is achieved by casting the calibration as a segment-based optimization problem that can be run on a small subset of informative segments. Consequently, the computational burden is limited as only a predefined number of segments is used. We also propose an efficient information-theoretic selection to identify such informative motion segments. In evaluations on a challenging dataset, we show our approach to significantly outperform state-of-the-art in terms of computational burden while maintaining a comparable accuracy.

Introduction

Visual-inertial tracking systems, which fuse data from cameras and Inertial Measurement Units (IMUs), are foundational to modern mobile robotics, AR/VR headsets, and smartphones. For these systems to estimate position accurately, sensors must be precisely calibrated. This includes intrinsic parameters (like camera focal length and IMU axis misalignment) and extrinsic parameters (the physical 3D transformation between the camera and the IMU).

While devices are calibrated in the factory, these parameters drift over time due to temperature changes, physical shocks, and general wear. Self-calibration "in the wild" is possible, but standard "full-batch" approaches that process all collected data are computationally prohibitive for battery-powered, resource-constrained consumer devices. This paper presents a method to achieve full-batch accuracy using only a fraction of the computational budget by selecting only the most "informative" motion segments.

Methodology

The proposed system operates as a background self-calibration module alongside a standard visual-inertial SLAM system. Instead of processing the entire data stream, the system partitions it into short motion segments and selects a sparse subset for optimization.

System Model and Parameters

The calibration targets the transformations between the camera (C), gyroscope (I), and accelerometer (A), as well as their respective intrinsic properties.

Visual-inertial system model and calibration parameters.

Informative Segment Selection

The core novelty lies in the information-theoretic selection process:

  1. Entropy-Based Scoring: The system evaluates the information content of each candidate segment using differential entropy. To keep this tractable on mobile processors, a rank-revealing QR decomposition approximates the marginal covariance of the calibration parameters, avoiding expensive matrix inversions.
  2. Parameter Grouping: To prevent strong signals (like high-dynamic motion) from overshadowing harder-to-observe parameters, the authors split parameters into three groups: IMU intrinsics, camera intrinsics, and extrinsics. Separate "top N" lists are maintained for each group.
  3. Sparsified Optimization: A non-linear least squares optimization is run only on the curated database of the most informative segments.
Entropy-based identification of informative motion segments along a trajectory.

Experimental Results

The approach was evaluated on 15 real-world datasets collected using a Google Tango tablet. The trajectories featured a mix of calm and highly dynamic movements designed to excite all rotational and translational degrees of freedom.

  • Massive Speedup: The proposed method completed calibration in an average of 31.2 seconds—roughly 6x faster than the full-batch approach and over 10x faster than comparable state-of-the-art segment-based methods.
  • Maintained Accuracy: Despite discarding the majority of the trajectory data, the estimated parameters showed insignificant deviation from the computationally heavy full-batch baseline.
  • Metric Validation: Ablation studies selecting the least informative segments resulted in higher estimation errors and variance, proving the entropy metric successfully identifies high-value calibration data.

Conclusion

This research demonstrates that visual-inertial systems do not require hours of data for calibration; they require the right kind of data. By mathematically identifying informative segments, the system achieves a 10x reduction in computational time without sacrificing accuracy. Future work will focus on formalizing the co-observability structure of parameters to further optimize segment grouping.

Summary updated

Abstract

Environmental conditions and external effects, such as shocks, have a significant impact on the calibration parameters of visual-inertial sensor systems. Thus long-term operation of these systems cannot fully rely on factory calibration. Since the observability of certain parameters is highly dependent on the motion of the device, using short data segments at device initialization may yield poor results. When such systems are additionally subject to energy constraints, it is also infeasible to use full-batch approaches on a big dataset and careful selection of the data is of high importance. In this paper, we present a novel approach for resource efficient self-calibration of visual-inertial sensor systems. This is achieved by casting the calibration as a segment-based optimization problem that can be run on a small subset of informative segments. Consequently, the computational burden is limited as only a predefined number of segments is used. We also propose an efficient information-theoretic selection to identify such informative motion segments. In evaluations on a challenging dataset, we show our approach to significantly outperform state-of-the-art in terms of computational burden while maintaining a comparable accuracy.
BibTeX
@inproceedings{schneider2017visual,
  author = {Thomas Schneider and Mingyang Li and Michael Burri and Juan I. Nieto and Roland Siegwart and Igor Gilitschenski},
  title = {{Visual-inertial self-calibration on informative motion segments}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2017},
  url = {https://openreview.net/forum?id=AkLK5CqRMQD},
}

Loading related content...

Figure