SE3ET: SE(3)-Equivariant Transformer for Low-Overlap Point Cloud Registration

Chien Erh Lin, Minghan Zhu, Maani Ghaffari

Abstract

Point cloud registration is a fundamental task in robotics and 3D vision, yet it remains challenging under conditions of low overlap and arbitrary initial rotations. We present SE3ET, a novel framework that leverages SE(3)-equivariant transformers to achieve robust alignment. By embedding geometric symmetry directly into the neural architecture, SE3ET achieves state-of-the-art performance and demonstrates exceptional generalization across diverse environments without the need for extensive data augmentation.

Introduction

Point cloud registration involves determining the relative transformation between two 3D scans, a critical component for autonomous navigation and mapping. Traditional deep learning approaches often struggle with two primary real-world hurdles: low overlap between scans (often as low as 10-30%) and sensitivity to initial orientations. While many models rely on rotation-invariant features, these often discard vital geometric information. SE3ET addresses these limitations by utilizing equivariant features that preserve geometric structure while ensuring consistent responses across all orientations.

Methodology

The SE3ET framework is built upon the principle of SE(3)-equivariance, where internal features rotate predictably in response to input rotations.

Equivariant Backbone

The model employs an Efficient SE(3)-Equivariant Point Network (E2PN) as its backbone. Unlike previous methods that use complex icosahedral groups, E2PN discretizes the rotation group into an octahedral group (24 rotations). This approach is computationally efficient and integrates seamlessly with voxel-based downsampling techniques.

Equivariant Transformer Design

Standard transformers typically ignore the geometric structure of 3D tensors. SE3ET introduces four specialized transformer designs—ESA, ICA, ACA, and RCA—that allow the attention mechanism to reason about spatial locations and orientations simultaneously. This ensures that the transformer layers maintain the equivariant property throughout the feature extraction process.

Coarse-to-Fine Alignment

The pipeline follows a hierarchical approach: it first identifies correspondences between "superpoints" (coarse patches) to establish a global alignment and then refines the transformation using dense point-to-point correspondences.

Experimental Evaluation

SE3ET was evaluated on several rigorous benchmarks:

  • 3DMatch & 3DLoMatch: Indoor RGB-D datasets, with 3DLoMatch specifically testing low-overlap (10-30%) scenarios.
  • Rotated Benchmarks: Harder variants of the indoor datasets where test pairs are subjected to arbitrary 3D rotations.
  • KITTI: A large-scale outdoor LiDAR dataset. A custom low-overlap version was created by selecting frames separated by at least 10 meters.

Results and Discussion

SE3ET demonstrates significant improvements in robustness and precision compared to existing state-of-the-art models.

Qualitative comparison of SE3ET against GeoTransformer across Easy, Moderate, and Challenging registration scenarios with large initial rotations.

Performance Metrics

On the Rotated 3DLoMatch benchmark, SE3ET-I achieved a Registration Recall of 70.8%, surpassing GeoTransformer (68.5%) and the equivariant model YOHO (65.9%). In outdoor settings (KITTI), SE3ET achieved a Relative Translation Error of just 4.7 cm and a Relative Rotation Error of 0.21°, the lowest among all compared methods.

Generalization and Efficiency

One of the most striking results is the model's zero-shot generalization capability. When trained on indoor data (3DMatch) and tested on outdoor data (KITTI), SE3ET maintained a 75.9% recall. In contrast, the non-equivariant RoITr model's performance dropped to 20.9%. Furthermore, the SE3ET-I2 variant remains computationally efficient, processing pairs in approximately 105ms with fewer parameters than many non-equivariant competitors.

Conclusion

By incorporating symmetry as a structural prior, SE3ET provides a robust solution for point cloud registration in challenging environments. The model's ability to handle arbitrary rotations and low overlap without exhaustive data augmentation marks a significant step forward for reliable robotic perception. Future work will focus on reducing the dependency on RANSAC for final alignment and integrating the framework into full SLAM systems.

Summary updated

Abstract

Partial point cloud registration is a challenging problem, especially when the robot undergoes a large transformation, causing a significant initial pose error and a low overlap. This work proposes exploiting equivariant learning from 3D point clouds to improve registration robustness. We propose SE3ET, an SE(3)-equivariant registration framework that employs equivariant point convolution and equivariant transformer design to learn expressive and robust geometric features. We tested the proposed registration method on indoor and outdoor benchmarks where the point clouds are under arbitrary transformations and low overlapping ratios. We also provide generalization tests and run-time performance.
BibTeX
@inproceedings{lin2024se,
  author = {Chien Erh Lin and Minghan Zhu and Maani Ghaffari},
  title = {{SE3ET: SE(3)-Equivariant Transformer for Low-Overlap Point Cloud Registration}},
  booktitle = {GRaM @ ICML},
  year = {2024},
  url = {https://openreview.net/forum?id=dDP6m3p9gj},
}
Figure