Robust Outlier Rejection for 3D Registration with Variational Bayes

Haobo Jiang, Zheng Dang, Zhen Wei, Jin Xie, Jian Yang, Mathieu Salzmann

1. Introduction

3D registration—the process of aligning multiple 3D scans into a unified coordinate system—is a foundational task in autonomous driving, robotics, and large-scale 3D reconstruction. The standard pipeline typically involves identifying "putative correspondences" between scans. However, these matches are frequently contaminated by a high volume of outliers (false matches) due to sensor noise, repetitive environmental structures, and low overlap between different viewpoints.

While deep learning methods like PointDSC have improved outlier rejection by utilizing non-local networks to enforce global spatial consistency, they often falter when the outlier ratio is extreme. In scenarios where only a small fraction (e.g., 5%) of matches are correct, spatial consistency becomes ambiguous, leading networks to converge on incorrect solutions. This paper introduces VBReg, a framework designed to handle such ambiguity through Variational Bayesian inference.

2. Methodology

VBReg shifts the paradigm from deterministic feature estimation to probabilistic modeling. The framework consists of two primary innovations: Variational Feature Learning and a Wilson score-based voting mechanism.

2.1 Variational Feature Learning

Traditional networks predict a single, fixed feature vector for each correspondence. In contrast, VBReg models these features as probability distributions characterized by a mean and variance. This allows the network to explicitly represent the uncertainty associated with each match.

During the training phase, the authors employ a "posterior" encoder that has access to ground-truth labels (identifying which points are true inliers). A "prior" encoder, which operates without labels, is trained to approximate this discriminative posterior distribution using Kullback–Leibler (KL) divergence. This strategy ensures that at inference time, the prior encoder produces features that are inherently more capable of distinguishing inliers from outliers, even without external guidance.

2.2 Voting-based Inlier Searching

To estimate the final transformation, many systems rely on RANSAC (Random Sample Consensus). However, RANSAC's stochastic nature can lead to instability in high-outlier environments. VBReg replaces this with a deterministic voting mechanism based on the Wilson score. By iteratively processing matches through the network, the system "votes" on the likelihood of a point being an inlier. This approach is theoretically more robust and provides a more stable estimate of the inlier set compared to traditional random sampling.

3. Experiments and Results

The authors evaluated VBReg across three diverse datasets: 3DMatch (indoor), 3DLoMatch (low-overlap indoor), and KITTI (outdoor LiDAR).

3.1 Performance on Low-Overlap Scenarios

The most significant gains were observed in the 3DLoMatch benchmark, where scan overlap is limited to 10%–30%. In these challenging settings, VBReg outperformed the previous state-of-the-art method, PointDSC, by a margin of 7.8% to 9.5% in Registration Recall when using a limited number of points (250-500).

3.2 Generalization and Efficiency

VBReg demonstrated strong performance across all benchmarks:

  • 3DMatch: Achieved a Registration Recall of 93.53%.
  • KITTI: Reached 98.92% recall, proving its efficacy in large-scale outdoor environments.
  • Computational Overhead: Despite the complexity of probabilistic modeling, the system remains efficient, processing correspondences in approximately 0.22 seconds. This makes it suitable for near real-time applications in robotics.

4. Discussion and Conclusion

The core insight of VBReg is that embracing uncertainty is more effective than attempting to force certainty in noisy data. By modeling long-range geometric dependencies as distributions, the network avoids being misled by clusters of "consistent" but incorrect outliers.

While VBReg is highly robust, it faces limitations in extremely sparse point clouds where the total number of correspondences is very low. Although the authors introduced a "conservative seed selection" strategy to mitigate this, sparse data remains an open challenge. Future research may explore integrating variational inference directly into the initial descriptor extraction phase to create a fully end-to-end probabilistic registration pipeline.

Summary updated

Abstract

Learning-based outlier (mismatched correspondence) rejection for robust 3D registration generally formulates the outlier removal as an inlier/outlier classification problem. The core for this to be successful is to learn the discriminative inlier/outlier feature representations. In this paper, we develop a novel variational non-local network-based outlier rejection framework for robust alignment. By reformulating the non-local feature learning with variational Bayesian inference, the Bayesian-driven long-range dependencies can be modeled to aggregate discriminative geometric context information for inlier/outlier distinction. Specifically, to achieve such Bayesian-driven contextual dependencies, each query/key/value component in our nonlocal network predicts a prior feature distribution and a posterior one. Embedded with the inlier/outlier label, the posterior feature distribution is label-dependent and discriminative. Thus, pushing the prior to be close to the discriminative posterior in the training step enables the features sampled from this prior at test time to model highquality long-range dependencies. Notably, to achieve effective posterior feature guidance, a specific probabilistic graphical model is designed over our non-local model, which lets us derive a variational low bound as our optimization objective for model training. Finally, we propose a voting-based inlier searching strategy to cluster the high-quality hypothetical inliers for transformation estimation. Extensive experiments on 3DMatch, 3DLoMatch, and KITTI datasets verify the effectiveness of our method. Code is available at https://github.com/Jiang-HB/VBReg.
BibTeX
@inproceedings{jiang2023robust,
  author = {Haobo Jiang and Zheng Dang and Zhen Wei and Jin Xie and Jian Yang and Mathieu Salzmann},
  title = {{Robust Outlier Rejection for 3D Registration with Variational Bayes}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2023},
}
Figure