One-Inlier is First: Towards Efficient Position Encoding for Point Cloud Registration

Fan Yang, Lin Guo, Zhi Chen, Wenbing TaoNeurIPS 2022 · OpenReview

Point cloud registration—aligning two 3D scans into a single coordinate system—is a foundational problem for autonomous driving and robotic mapping. Recently, researchers have turned to Transformers to solve this, as they excel at finding correspondences between points. However, Transformers are inherently "order-agnostic," meaning they don't know where a point is in space unless provided with Position Encoding (PE).

The challenge is that point cloud registration is a "two-instance" task. The two scans are captured from different viewpoints, meaning their coordinate systems are completely uncorrelated. If you just plug in raw coordinates, the Transformer gets confused because the same physical object has different "addresses" in each scan. Previous solutions tried to solve this by calculating the relative geometry between all pairs of points, but this is computationally expensive and memory-intensive.

The "One-Inlier" Solution (OIF-PCR)

Virtual Inlier Concept

The authors propose OIF-PCR (One-Inlier based Position Encoding for Point Cloud Registration). Their core insight is that you don't need to calculate complex relationships between every point; you just need one reliable shared reference point (an inlier). By normalizing two different point clouds against this common anchor, the model achieves state-of-the-art accuracy while using only a fraction of the memory required by previous heavyweights.

How It Works

OIF-PCR Architecture

The OIF-PCR framework operates through a joint optimization process:

  1. Inlier Learning: Instead of picking a random point, the model uses a differentiable optimal transport layer to find the best matches between the two clouds. It selects the "top-k" most confident matches and averages them to create a virtual correspondence.
  2. Position Embedding: This virtual correspondence acts as a shared origin. The model subtracts this reference point from every other point in its respective cloud. Because rigid transformations preserve distances, points that correspond to each other will now have similar vectors relative to this shared "anchor."
  3. Joint Optimization: The authors solve the "chicken-and-egg" problem of needing good features to find an inlier, and an inlier to encode positions, with an iterative process. The model finds an inlier, aligns the clouds, updates the features, and repeats.
  4. Progressive Alignment: The model gradually narrows the "position gap" between the two clouds over several iterations rather than attempting a perfect alignment in one shot.

Key Results and Efficiency

The authors evaluated OIF-PCR on three standard benchmarks: 3DMatch (indoor, high overlap), 3DLoMatch (indoor, low overlap), and KITTI (outdoor driving).

  • Accuracy: On the challenging 3DLoMatch dataset, OIF-PCR achieved a 76.1% Registration Recall, outperforming the previous state-of-the-art (GeoTransformer at 75.0%). On the KITTI dataset, it achieved the best results across all metrics, including a Relative Rotation Error (RRE) of just 0.23°.
  • Efficiency: OIF-PCR shines in its resource management. It uses roughly 4.85 GB of GPU memory, compared to 12.73 GB for GeoTransformer—a massive 60% reduction. It also boasts the fastest total runtime among end-to-end methods (0.555s vs. 1.99s).
  • Robustness: By creating a "virtual" inlier from the top-k matches rather than picking a single physical point, the model avoids the instability of choosing a single outlier by mistake.

Limitations and Future Work

While highly efficient, the method relies on a coarse-to-fine strategy. If the initial coarse matches are completely wrong, the "one-inlier" logic fails to provide a helpful anchor. Furthermore, if the overlapping region between two scans is extremely small or contains ambiguous, repetitive structures (like a long, featureless hallway), the model may struggle to find that first crucial inlier. Future work may explore using point cloud completion to "fill in" missing parts of the scans, effectively expanding the overlapping region before registration.

Summary updated

Abstract

Transformer architecture has shown great potential for many visual tasks, including point cloud registration. As an order-aware module, position encoding plays an important role in Transformer architecture applied to point cloud registration task. In this paper, we propose OIF-PCR, a one-inlier based position encoding method for point cloud registration network. Specifically, we first find one correspondence by a differentiable optimal transport layer, and use it to normalize each point for position encoding. It can eliminate the challenges brought by the different reference frames of two point clouds, and mitigate the feature ambiguity by learning the spatial consistency. Then, we propose a joint approach for establishing correspondence and position encoding, presenting an iterative optimization process. Finally, we design a progressive way for point cloud alignment and feature learning to gradually optimize the rigid transformation. The proposed position encoding is very efficient, requiring only a small addition of memory and computing overhead. Extensive experiments demonstrate the proposed method can achieve competitive performance with the state-of-the-art methods in both indoor and outdoor scenes.
BibTeX
@inproceedings{yang2022one,
  author = {Fan Yang and Lin Guo and Zhi Chen and Wenbing Tao},
  title = {{One-Inlier is First: Towards Efficient Position Encoding for Point Cloud Registration}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2022},
  url = {https://openreview.net/forum?id=19MmorTQhho},
}
Figure