TL;DR
This paper introduces a novel "graybox" system identification method for jointly calibrating low-cost IMUs and rolling shutter cameras, a common pairing for mobile robot localization. Crucially, it also estimates the IMU's unknown noise parameters, leading to more accurate and stable calibration results, especially in challenging real-world scenarios.
What Problem Does This Paper Address?
Accurate calibration of sensors is paramount for reliable visual-inertial navigation systems, especially in GPS-denied environments. While IMUs and cameras are complementary and cost-effective, their joint calibration is complex. Existing methods often calibrate sensors separately, leading to error propagation, or assume known noise parameters for the IMU, which are difficult to determine accurately for low-cost devices. This paper tackles the challenge of jointly calibrating these heterogeneous sensors while simultaneously identifying the critical, often unknown, noise characteristics of the IMU.
Methods
The core of the proposed method is a "graybox" system identification approach. This involves a two-stage process:
- Filtering and Noise Initialization: An Unscented Kalman Filter (UKF) is used to initialize calibration parameters and estimate IMU motion (position, velocity, orientation). Simultaneously, a grid search is employed to find initial estimates for the IMU's noise parameters.
- Joint Calibration and Noise Estimation: A nonlinear optimization problem is formulated to minimize a cost function that considers both the residuals between predicted and observed measurements and the state error covariances from the UKF. This "noise-aware" cost function allows for the joint estimation of calibration parameters and noise densities.
- Refinement: A final refinement step uses a conventional graybox method (minimizing only residuals) to further improve calibration accuracy, with the UKF now solely focused on estimating motion states.
The novelty lies in the joint estimation of calibration and noise parameters, and the specific cost function that incorporates state error covariances to guide the noise estimation.
Datasets
The paper evaluates its method using:
- Synthetic Data: Generated motion information and calibration pattern points to validate accuracy and stability through Monte-Carlo simulations.
- Real-World Data:
- A self-designed experimental setup with two rolling shutter cameras and a low-cost IMU.
- A commercial smartphone ("Samsung Galaxy Alpha") with a rolling shutter camera and a low-cost IMU.
- Outdoor driving sequences for localization accuracy evaluation.
Key Insights and Ideas
- Noise Matters: Accurately estimating IMU noise parameters is crucial for achieving robust and stable calibration, especially for low-cost sensors where this information is often unavailable.
- Joint Estimation is Key: Calibrating intrinsic and extrinsic parameters simultaneously, along with noise parameters, leads to more accurate and practical results than separate calibration approaches.
- "Noise-Aware" Cost Function: Incorporating state error covariances into the cost function is essential for preventing the optimizer from over-fitting to measurements and producing unrealistically large noise estimates.
- Two-Stage Approach for Efficiency: Dividing the calibration into filtering (initialization) and optimization (refinement) steps significantly speeds up the process compared to single-step optimization.
- Off-the-Shelf Viability: The method demonstrates strong performance on commercial smartphones, highlighting its practical applicability for consumer-grade devices.
Key Results
- Accuracy: The proposed method achieved accurate calibration parameters, with estimates consistently within the error bounds of ground truth in synthetic experiments.
- Stability: The method demonstrated stable estimates, with small standard deviations, particularly in real-world experiments.
- Noise Identification Effectiveness: Experiments showed that estimating IMU noise parameters significantly improved calibration accuracy and stability, especially when compared to using fixed, assumed noise values. For instance, translation estimates without noise identification were inaccurate and unstable.
- Runtime Improvement: The proposed method was significantly faster than existing methods (e.g., Kalibr), achieving a speedup of approximately 73.3% in runtime.
- Reduced Motion Drift: In visual-inertial SLAM, the proposed method reduced IMU drift by 19.2% compared to the Kalibr method.
- Improved Localization: Real-world outdoor localization experiments showed the proposed method's trajectory estimates were closer to the ground truth than those from the Kalibr method, with less scale drift over longer trajectories.
Limitations and Future Work
- Bias States: The paper mentions that bias states are treated as constant parameters, unlike some other methods that consider them time-varying. This simplification is justified by the short calibration time but could be a limitation for longer-term deployments.
- Assumptions: The method assumes a rigid connection between the IMU and the camera.
- Future Work: The authors suggest that the estimated noise parameters can be used for other sensor fusion algorithms, implying potential for further integration and application in downstream tasks like visual-inertial SLAM. The robustness to different types of IMUs and cameras could also be explored further.
Summary updated