TL;DR
This paper introduces ROVO, a robust visual odometry system designed for wide-baseline, wide-field-of-view (FOV) fisheye camera rigs. By combining a novel hybrid projection model, a multi-view P3P RANSAC algorithm, and online extrinsic calibration, ROVO achieves accurate and stable ego-motion estimation even in challenging, dynamic environments.
What Problem Does This Paper Address?
Estimating a robot's precise movement and orientation (ego-motion) is crucial for autonomous navigation. While visual odometry (VO) systems using cameras are cost-effective and information-rich, traditional systems face limitations. Monocular systems can't determine true scale, and stereo systems often have a limited FOV, making them susceptible to dynamic objects and occlusions. Wide-baseline, wide-FOV fisheye cameras offer a solution by providing a full 360° view and better stereo triangulation, but they introduce new challenges: severe image distortion at the periphery and the potential for the camera rig itself to deform due to vibrations or temperature changes, invalidating the assumption of a rigid structure. This paper tackles these challenges to enable reliable VO in complex scenarios.
Methods
ROVO enhances the standard VO pipeline with three key innovations:
- Hybrid Projection Model: To overcome the distortion inherent in wide-FOV fisheye lenses, this model warps images using a combination of perspective and cylindrical projections. This creates overlapping regions that are more perspective-like, improving feature matching. Crucially, the warped image remains continuous and smooth, essential for tracking.
- Multi-view P3P RANSAC: Instead of relying on a single view, this algorithm extends the traditional P3P RANSAC to consider feature matches across multiple views. This allows for more robust pose estimation by effectively counting inliers from all available camera perspectives, making it resilient to dynamic objects.
- Online Extrinsic Calibration: Recognizing that camera rigs can deform, this component seamlessly integrates into the backend optimization. It continuously updates the relative poses (extrinsic parameters) between cameras, automatically correcting for changes caused by shocks or vibrations, thereby improving accuracy and robustness.
Datasets
The authors evaluated ROVO using two types of datasets:
- Synthetic Datasets: Four urban sequences were rendered using Blender, simulating different structures, lighting conditions, weather, and camera configurations (four simulated fisheye cameras with 220° FOV). These datasets provided ground-truth poses and camera parameters for quantitative evaluation.
- Real-World Datasets: The system was tested on real sequences captured by a four-camera rig mounted on a vehicle. These datasets presented challenges like harsh illumination, dynamic traffic, and narrow streets.
Key Insights and Ideas
- Distortion is the Enemy of Matching: The paper highlights how the extreme distortion in fisheye lenses significantly degrades feature matching. The hybrid projection model is a clever way to "un-distort" the relevant overlapping areas without the computational cost or information loss of full unwrapping.
- Multi-view is Key for Robustness: In dynamic environments, relying on a single view for pose estimation is a recipe for disaster. The multi-view P3P RANSAC effectively leverages redundancy across cameras to filter out outliers caused by moving objects.
- Rigidity is an Assumption, Not a Guarantee: The online extrinsic calibration is a critical component for real-world applications. It acknowledges that physical systems are not perfectly rigid and builds in a mechanism to adapt to these changes, which is vital for long-term accuracy.
Key Results
- Improved Accuracy: ROVO, especially with online extrinsic calibration (OnlineExt), significantly reduces translation Root Mean Squared Error (RMSE) compared to systems without it (NoisyExt), often performing close to ground-truth (GTExt) on synthetic datasets. For instance, in the "Static Urban" sequence, OnlineExt achieved an RMSE of 5.0m, a substantial improvement over NoisyExt's 83.17m.
- Enhanced Matching Performance: The hybrid projection model led to a more than 3x increase in the inlier ratio for inter-view matching and reduced descriptor distance, directly contributing to better VO performance.
- Robustness in Dynamic Scenes: The system demonstrated its ability to reject outliers from moving objects and maintain accurate trajectories in challenging real-world sequences with heavy traffic.
- Fast Convergence of Calibration: Online extrinsic calibration quickly converged to ground-truth values within approximately 100 frames in both synthetic and real datasets, showcasing its effectiveness.
Limitations and Future Work
- Computational Cost: While the paper doesn't explicitly detail computational load, multi-camera systems and advanced optimization generally imply higher computational requirements than monocular systems.
- Loop Closing: The paper notes that their system does not use loop closing, which is a common technique in SLAM for correcting accumulated drift over long trajectories. Incorporating loop closing could further enhance long-term accuracy.
- Future Work: The authors suggest that their system's components are effective and could be applied to other multi-camera VO/SLAM systems. The public release of their datasets and implementation is also a step towards future research.
Summary updated