Adversarial point set registration

Sergei Divakov, Ivan V. OseledetsarXiv 2018 · OpenReview

Abstract: We present a novel approach to point set registration which is based on one-shot adversarial learning. The idea of the algorithm is inspired by recent successes of generative adversarial networks. Treating the point clouds as three-dimensional probability distributions, we develop a one-shot adversarial optimization procedure, in which we train a critic neural network to distinguish between source and target point sets, while simultaneously learning the parameters of the transformation to trick the critic into confusing the points. In contrast to most existing algorithms for point set registration, ours does not rely on any correspondences between the point clouds. We demonstrate the performance of the algorithm on several challenging benchmarks and compare it to the existing baselines.

Introduction

Point set registration—the process of finding the best rigid transformation (rotation and translation) to align two 3D point clouds—is a cornerstone of computer vision, essential for applications ranging from autonomous navigation to robotic manipulation. Traditional registration techniques generally fall into two categories: local methods and global methods. Local methods, such as the Iterative Closest Point (ICP) algorithm, are computationally efficient and precise but require the point clouds to be roughly aligned initially to avoid local minima. Global methods attempt to find an initial alignment but often rely on hand-crafted geometric features or explicit point-to-point correspondences.

The "correspondence problem" remains a significant bottleneck. When point clouds are noisy, have varying densities, or lack distinct geometric features, finding accurate matches between individual points becomes unreliable. This paper proposes a novel framework that aligns point clouds by treating them as probability distributions, bypassing the need for explicit correspondences entirely.

Methodology: One-Shot Adversarial Learning

The core of our approach is an adversarial game inspired by Generative Adversarial Networks (GANs). However, unlike standard GANs that learn to generate new data from a distribution, our model performs "one-shot" optimization to find a specific geometric transformation for a given pair of point clouds.

The adversarial registration pipeline. A critic network learns to distinguish between target points and transformed source points, while the transformation parameters M are optimized to align the sets.

The Critic and the Generator

In this setup, a shallow, four-layer neural network acts as the Critic. Its goal is to distinguish between points from the "target" cloud and the "transformed source" cloud. Simultaneously, a Generator—represented here as a rigid transformation $M \in SE(3)$—attempts to move the source points so precisely that the Critic can no longer differentiate them from the target points.

Technical Implementation

To ensure stable and effective training, several key technical choices were made:

  • Wasserstein Metric: We utilize the Wasserstein-1 distance (Earth-Mover’s Distance) as the loss function, implemented with a gradient penalty (WGAN-GP). This provides a smooth gradient for the optimizer, even when the point clouds are initially far apart.
  • Lie Algebra Parametrization: Rotations are represented using the exponential mapping of $so(3)$ (axis-angle). This allows for unconstrained optimization, which is more stable for stochastic gradient descent than using quaternions.
  • Shift Augmentation: To prevent the model from conflating rotation and translation, we introduce a "shift augmentation" trick. During training, both point sets are randomly shifted and rescaled, forcing the Critic to focus on the underlying shape and orientation rather than absolute spatial coordinates.

Experimental Evaluation

We evaluated the proposed method against several benchmarks, including the classic Stanford Bunny for synthetic tests and a custom car dataset for real-world challenges. The car dataset involved matching a dense depth scan (over 400,000 points) against a sparse CAD model (~5,000 vertices), representing a highly ill-posed problem due to the massive density mismatch.

Key Results and Robustness

The adversarial approach demonstrated significant advantages over traditional baselines:

  • Initial Pose Robustness: While ICP typically fails if the initial rotation exceeds 60 degrees, our method maintained a high success rate even with 180-degree initial rotations.
  • Noise Tolerance: The distribution-based approach is naturally robust to sampling noise. As shown in our comparative tests, the adversarial method maintains high performance at noise levels where ICP's success rate drops precipitously.
Robustness to noise: The proposed method (blue) significantly outperforms the Iterative Closest Point (ICP) algorithm (orange) as Gaussian noise intensity increases.
  • Real-World Accuracy: On the challenging car-matching task, the algorithm achieved a rotation error of just 1.8 degrees, significantly outperforming feature-based methods like Fast Point Feature Histograms (FPFH).

Discussion and Limitations

While the adversarial approach offers superior robustness, it faces certain limitations. The method is currently sensitive to significant outliers; if one cloud contains substantial data that does not exist in the other, the distribution matching can fail. Additionally, because the method requires training a neural network from scratch for every registration task, it is currently slower than traditional geometric pipelines, making it less suitable for real-time SLAM applications in its current form.

Future work will focus on improving computational efficiency and enhancing robustness to partial overlaps. Nevertheless, this "one-shot" optimization style demonstrates that framing registration as a distribution matching problem is a viable and powerful alternative to correspondence-based methods.

Summary updated

Abstract

We present a novel approach to point set registration which is based on one-shot adversarial learning. The idea of the algorithm is inspired by recent successes of generative adversarial networks. Treating the point clouds as three-dimensional probability distributions, we develop a one-shot adversarial optimization procedure, in which we train a critic neural network to distinguish between source and target point sets, while simultaneously learning the parameters of the transformation to trick the critic into confusing the points. In contrast to most existing algorithms for point set registration, ours does not rely on any correspondences between the point clouds. We demonstrate the performance of the algorithm on several challenging benchmarks and compare it to the existing baselines.
BibTeX
@misc{divakov2018adversarial,
  author = {Sergei Divakov and Ivan V. Oseledets},
  title = {{Adversarial point set registration}},
  year = {2018},
  eprint = {1811.08139},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1811.08139},
}
Figure