TL;DR
This paper introduces a novel self-supervised method for estimating the relative pose between camera views in road scenes. By leveraging the inherent planarity of the road surface, they develop a simpler, more efficient approach that avoids the need for dense depth estimation, achieving competitive results with state-of-the-art methods.
What Problem Does This Paper Address?
Estimating the relative pose (how one camera view is positioned and oriented with respect to another) is crucial for many computer vision tasks, especially visual odometry (VO) in autonomous vehicles. While deep learning methods have shown promise, they often require vast amounts of labeled data, which is expensive and difficult to collect. Existing self-supervised methods typically try to learn both depth and pose simultaneously, which can be an ill-posed problem leading to suboptimal pose estimation. These methods often rely on complex architectures and can be sensitive to errors in depth estimation. The authors aim to address the challenge of accurate and efficient self-supervised relative pose estimation, particularly for road scenes, by exploiting geometric priors.
Methods
The core of the proposed method lies in exploiting the approximate planarity of the road surface. Instead of learning dense depth, they focus on estimating a 9D ground-relative pose. This parameterization simplifies the problem and allows for a self-supervision signal to be derived from cross-projecting images using a homography.
The method involves two main stages:
- Pre-training with Perceptual Loss: The network is initially trained using a perceptual loss, which compares deep features of the warped and target images. This stage also incorporates a prior loss to enforce known camera height, roll, and pitch distributions from dataset calibrations, helping to resolve scale ambiguity inherent in planar homography.
- Post-training with Homography Estimation Module (HEM): This stage introduces a non-differentiable module that refines the pose. It uses pre-trained optical flow and semantic segmentation networks to isolate road plane pixels. RANSAC is then applied to these pixels to robustly estimate a homography. This estimated homography is decomposed to obtain refined camera-relative pose, which is then used as pseudo-labels to supervise the network. This HEM can be applied during training and/or at test time.
The authors also highlight the use of a geometric matching architecture (inspired by Rocco et al. [25]) for feature extraction and matching, which they claim is more effective for capturing geometric perspective than simple concatenation of input images.
Datasets
The primary dataset used for training and evaluation is the KITTI VO dataset [12]. The authors train on the raw dataset sequences and evaluate on sequences 09 and 10, as well as benchmark test sets. They do not introduce a new dataset.
Key Insights and Ideas
- Leveraging Road Planarity: The key insight is that the approximate planarity of road surfaces is a strong geometric prior that can significantly simplify self-supervised pose estimation, eliminating the need for complex depth estimation.
- 9D Ground-Relative Pose: Parameterizing pose relative to the ground plane (9 degrees of freedom) is more general and powerful than fixed camera-relative 6 DoF poses, allowing for easier extraction of various transformations.
- Homography as a Supervision Signal: Using homographies derived from ground-relative pose for cross-projection provides a robust self-supervision signal, especially for planar scenes.
- Non-Differentiable Model-Fitting in the Loop: Integrating a non-differentiable homography estimation module (HEM) for refinement and pseudo-label generation is a novel approach to self-supervision, improving robustness and tackling dataset bias.
- Perceptual Loss for Initialization: Using a perceptual loss for pre-training provides a wider basin of convergence and avoids issues with illumination assumptions required for pixel-level losses.
Key Results
- The proposed method achieves competitive results on the KITTI VO benchmark, performing comparably to state-of-the-art self-supervised methods that often rely on dense depth estimation.
- The HEM refinement significantly improves performance, especially when applied during training and at test time.
- Qualitative results show increasing refinement with the HEM, with accurate pose estimation even in challenging scenarios with rapid gradient changes.
- The method is shown to be flexible and unconstrained, handling arbitrary pose changes and being easy to train and use.
- Using the HEM with a standard Monodepth2 [13] pose network resulted in significantly worse performance compared to their custom geometric matching network, suggesting the importance of the network architecture.
Limitations and Future Work
- Illumination Issues: The HEM can be prone to issues with dynamic shadows or glare, which can lead to miscorrespondences and affect accuracy (as seen in the final examples of Fig. 5).
- Planar Assumption: While the method can handle scenes with gradient changes by approximating them with planar patches, it is fundamentally based on a planar road model.
- Future Work: The authors plan to extend the planar constraint to a more complex geometrical model and to utilize richer semantic understanding. They also suggest exploring the use of their ground-relative pose for estimating road depth.
Summary updated