Abstract
Establishing dense correspondences between a pair of images is an important and general problem. However, dense flow estimation is often inaccurate in the case of large displacements or homogeneous regions. For most applications and down-stream tasks, such as pose estimation, image manipulation, or 3D reconstruction, it is crucial to know when and where to trust the estimated matches. In this work, we aim to estimate a dense flow field relating two images, coupled with a robust pixel-wise confidence map indicating the reliability and accuracy of the prediction. We develop a flexible probabilistic approach that jointly learns the flow prediction and its uncertainty. In particular, we parametrize the predictive distribution as a constrained mixture model, ensuring better modelling of both accurate flow predictions and outliers. Moreover, we develop an architecture and training strategy tailored for robust and generalizable uncertainty prediction in the context of self-supervised training. Our approach obtains state-of-the-art results on multiple challenging geometric matching and optical flow datasets. We further validate the usefulness of our probabilistic confidence estimation for the task of pose estimation.
Introduction
Finding "dense correspondences"—matching every pixel in one image to its counterpart in another—is a cornerstone of computer vision. While significant progress has been made in optical flow for consecutive video frames, the more general problem of "geometric matching" remains challenging. When images are captured from radically different viewpoints, at different times, or contain large homogeneous regions like clear skies or plain walls, standard algorithms frequently fail.
A major issue is that many networks produce "over-confident" predictions in regions where no valid match exists. For downstream applications like autonomous driving or 3D mapping, an incorrect match can be more detrimental than no match at all. PDC-Net (Probabilistic Dense Correspondence Network) addresses this by learning to estimate dense flow while simultaneously quantifying the reliability of its own predictions.
Methodology
PDC-Net treats flow estimation as a probabilistic regression problem. Instead of outputting a simple 2D vector for each pixel, the network predicts a conditional probability distribution.
Constrained Mixture Model (CMM)
Rather than relying on a single distribution (e.g., a Gaussian), the authors employ a mixture of Laplace distributions. This "constrained" mixture model assigns specific components to model highly accurate inliers and others to handle large-error outliers. This architectural choice reflects the naturally bimodal nature of error distributions in dense matching, where a prediction is typically either very accurate or a complete miss.
Uncertainty Architecture
The network features a specialized "Correlation Uncertainty Module." Unlike standard decoders that may over-smooth results by aggregating information from neighbors, this module processes 2D slices of the correlation volume independently. This forces the network to assess match reliability based on local appearance and distinctness, preventing the "hallucination" of matches in featureless regions.
Self-Supervised Training
To ensure the model learns to be uncertain in ambiguous regions, the authors introduced small, local perturbations to synthetic training data. This strategy prevents the network from simply interpolating flow in blank regions, teaching it to flag areas where local appearance is insufficient for tracking.
Experimental Results
The model was evaluated across several challenging benchmarks, including MegaDepth (extreme viewpoint changes), RobotCar (varying lighting/weather), and KITTI (optical flow for autonomous driving).
- State-of-the-Art Accuracy: PDC-Net achieved record-breaking performance on MegaDepth and KITTI-2015, even outperforming specialized optical flow models like RAFT.
- Efficiency: PDC-Net is significantly faster than previous leading methods; for instance, it is 12.2 times faster than RANSAC-Flow while maintaining higher geometric accuracy.
- Downstream Utility: In pose estimation tasks on the YFCC100M dataset, using the predicted confidence maps to filter matches nearly doubled the Mean Average Precision (mAP) compared to non-probabilistic baselines.
Conclusion
PDC-Net demonstrates that joint learning of flow and uncertainty is key to robust geometric matching. By explicitly modeling outliers through a constrained mixture model, the network provides a functional tool for filtering unreliable data in complex pipelines like 3D reconstruction. While extreme viewpoint changes remain a challenge, the probabilistic framework significantly improves the reliability of dense correspondence estimation.
Summary updated