ROVO: Robust Omnidirectional Visual Odometry for Wide-baseline Wide-FOV Camera Systems

Hochang Seok, Jongwoo Lim

TL;DR

This paper introduces ROVO, a robust visual odometry system designed for wide-baseline, wide-field-of-view (FOV) fisheye camera rigs. By combining a novel hybrid projection model, a multi-view P3P RANSAC algorithm, and online extrinsic calibration, ROVO achieves accurate and stable ego-motion estimation even in challenging, dynamic environments.

What Problem Does This Paper Address?

Estimating a robot's precise movement and orientation (ego-motion) is crucial for autonomous navigation. While visual odometry (VO) systems using cameras are cost-effective and information-rich, traditional systems face limitations. Monocular systems can't determine true scale, and stereo systems often have a limited FOV, making them susceptible to dynamic objects and occlusions. Wide-baseline, wide-FOV fisheye cameras offer a solution by providing a full 360° view and better stereo triangulation, but they introduce new challenges: severe image distortion at the periphery and the potential for the camera rig itself to deform due to vibrations or temperature changes, invalidating the assumption of a rigid structure. This paper tackles these challenges to enable reliable VO in complex scenarios.

Methods

ROVO enhances the standard VO pipeline with three key innovations:

  1. Hybrid Projection Model: To overcome the distortion inherent in wide-FOV fisheye lenses, this model warps images using a combination of perspective and cylindrical projections. This creates overlapping regions that are more perspective-like, improving feature matching. Crucially, the warped image remains continuous and smooth, essential for tracking.
  2. Multi-view P3P RANSAC: Instead of relying on a single view, this algorithm extends the traditional P3P RANSAC to consider feature matches across multiple views. This allows for more robust pose estimation by effectively counting inliers from all available camera perspectives, making it resilient to dynamic objects.
  3. Online Extrinsic Calibration: Recognizing that camera rigs can deform, this component seamlessly integrates into the backend optimization. It continuously updates the relative poses (extrinsic parameters) between cameras, automatically correcting for changes caused by shocks or vibrations, thereby improving accuracy and robustness.

Datasets

The authors evaluated ROVO using two types of datasets:

  • Synthetic Datasets: Four urban sequences were rendered using Blender, simulating different structures, lighting conditions, weather, and camera configurations (four simulated fisheye cameras with 220° FOV). These datasets provided ground-truth poses and camera parameters for quantitative evaluation.
  • Real-World Datasets: The system was tested on real sequences captured by a four-camera rig mounted on a vehicle. These datasets presented challenges like harsh illumination, dynamic traffic, and narrow streets.

Key Insights and Ideas

  • Distortion is the Enemy of Matching: The paper highlights how the extreme distortion in fisheye lenses significantly degrades feature matching. The hybrid projection model is a clever way to "un-distort" the relevant overlapping areas without the computational cost or information loss of full unwrapping.
  • Multi-view is Key for Robustness: In dynamic environments, relying on a single view for pose estimation is a recipe for disaster. The multi-view P3P RANSAC effectively leverages redundancy across cameras to filter out outliers caused by moving objects.
  • Rigidity is an Assumption, Not a Guarantee: The online extrinsic calibration is a critical component for real-world applications. It acknowledges that physical systems are not perfectly rigid and builds in a mechanism to adapt to these changes, which is vital for long-term accuracy.

Key Results

  • Improved Accuracy: ROVO, especially with online extrinsic calibration (OnlineExt), significantly reduces translation Root Mean Squared Error (RMSE) compared to systems without it (NoisyExt), often performing close to ground-truth (GTExt) on synthetic datasets. For instance, in the "Static Urban" sequence, OnlineExt achieved an RMSE of 5.0m, a substantial improvement over NoisyExt's 83.17m.
  • Enhanced Matching Performance: The hybrid projection model led to a more than 3x increase in the inlier ratio for inter-view matching and reduced descriptor distance, directly contributing to better VO performance.
  • Robustness in Dynamic Scenes: The system demonstrated its ability to reject outliers from moving objects and maintain accurate trajectories in challenging real-world sequences with heavy traffic.
  • Fast Convergence of Calibration: Online extrinsic calibration quickly converged to ground-truth values within approximately 100 frames in both synthetic and real datasets, showcasing its effectiveness.

Limitations and Future Work

  • Computational Cost: While the paper doesn't explicitly detail computational load, multi-camera systems and advanced optimization generally imply higher computational requirements than monocular systems.
  • Loop Closing: The paper notes that their system does not use loop closing, which is a common technique in SLAM for correcting accumulated drift over long trajectories. Incorporating loop closing could further enhance long-term accuracy.
  • Future Work: The authors suggest that their system's components are effective and could be applied to other multi-camera VO/SLAM systems. The public release of their datasets and implementation is also a step towards future research.

Summary updated

Abstract

In this paper we propose a robust visual odometry system for a wide-baseline camera rig with wide field-of-view (FOV) fisheye lenses, which provides full omnidirectional stereo observations of the environment. For more robust and accurate ego-motion estimation we adds three components to the standard VO pipeline, 1) the hybrid projection model for improved feature matching, 2) multi-view P3P RANSAC algorithm for pose estimation, and 3) online update of rig extrinsic parameters. The hybrid projection model combines the perspective and cylindrical projection to maximize the overlap between views and minimize the image distortion that degrades feature matching performance. The multi-view P3P RANSAC algorithm extends the conventional P3P RANSAC to multi-view images so that all feature matches in all views are considered in the inlier counting for robust pose estimation. Finally the online extrinsic calibration is seamlessly integrated in the backend optimization framework so that the changes in camera poses due to shocks or vibrations can be corrected automatically. The proposed system is extensively evaluated with synthetic datasets with ground-truth and real sequences of highly dynamic environment, and its superior performance is demonstrated.
BibTeX
@inproceedings{seok2019rovo,
  author = {Hochang Seok and Jongwoo Lim},
  title = {{ROVO: Robust Omnidirectional Visual Odometry for Wide-baseline Wide-FOV Camera Systems}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2019},
  url = {https://openreview.net/forum?id=yoD3JaiqBg},
}
Figure