Abstract
We propose RPSRNet - a novel end-to-end trainable deep neural network for rigid point set registration. For this task, we use a novel $2^D$-tree representation for the input point sets and a hierarchical deep feature embedding in the neural network. An iterative transformation refinement module in our network boosts the feature matching accuracy in the intermediate stages. We achieve an inference speed of 12-15ms to register a pair of input point clouds as large as 250K. Extensive evaluation on (i) KITTI LiDAR odometry and (ii) ModelNet-40 datasets shows that our method outperforms prior state-of-the-art methods -- e.g., on the KITTI data set, DCP-v2 by 1.3 and 1.5 times, and PointNetLK by 1.8 and 1.9 times better rotational and translational accuracy respectively. Evaluation on ModelNet40 shows that RPSRNet is more robust than other benchmark methods when the samples contain a significant amount of noise and other disturbances. RPSRNet accurately registers point clouds with non-uniform sampling densities, e.g., LiDAR data, which cannot be processed by many existing deep-learning-based registration methods.
Introduction
Rigid Point Set Registration (RPSR) is a cornerstone of modern robotics and computer vision, essential for tasks like autonomous vehicle navigation (LiDAR odometry) and 3D medical imaging. The objective is to align a "source" point cloud to a "target" by determining the optimal rotation ($R$) and translation ($t$).
Existing solutions often face a trade-off between detail and efficiency. Classical methods like the Iterative Closest Point (ICP) algorithm are computationally expensive and sensitive to initial guesses. Meanwhile, deep learning approaches typically rely on memory-intensive voxel grids or point-based architectures that require aggressive downsampling, which discards vital geometric information. RPSRNet addresses these challenges by utilizing a hierarchical Barnes-Hut $2^D$-tree representation, allowing for high-resolution registration with remarkable speed.
Methodology
Barnes-Hut $2^D$-Tree Representation
The core innovation of RPSRNet is its use of an octree-based subdivision (a $2^D$-tree where $D=3$). Unlike standard voxel grids, this representation only stores non-empty regions of space, making it highly memory-efficient. Each node in the tree stores the Center of Mass (CoM) of the points within it and their Inverse Density (ID). This allows the network to leverage the inherent sparsity of point clouds as a computational advantage.
Hierarchical Feature Encoder (HFE)
The network employs a Hierarchical Feature Encoder that utilizes "Tree-Convolutions." For every non-empty node, it performs a 1D convolution across the node and its 26 neighbors at the same depth. This multi-scale approach enables the network to learn features ranging from fine geometric details to global structures. Furthermore, by fusing positional features with density features via a Hadamard product, the network becomes "density adaptive," allowing it to handle the non-uniform sampling densities common in LiDAR data.
Iterative Refinement and Transformer Module
Rather than a single-pass estimation, RPSRNet uses an iterative refinement process. At each step, a Transformer network learns contextual residuals between the source and target embeddings. A differentiable Singular Value Decomposition (SVD) module then regresses the transformation parameters. This iterative loop, illustrated below, allows the network to progressively sharpen the alignment.
Experimental Results
Accuracy and Robustness
RPSRNet was evaluated against benchmarks like ModelNet40 and KITTI LiDAR Odometry. On the KITTI dataset, RPSRNet outperformed the previous state-of-the-art (DCP-v2) by 1.3x in rotational accuracy and 1.5x in translational accuracy. In stress tests on ModelNet40, the network remained stable even when 40% of the input data was replaced with Gaussian noise, demonstrating superior robustness compared to PointNetLK.
Real-Time Performance
The architecture achieves an inference speed of approximately 12–15 ms for point cloud pairs as large as 250,000 points. This efficiency is largely due to the tree-based representation, which ignores empty space; the authors report that for sparse clouds, over 50% of tree cells are typically empty, leading to significant computational savings.
Conclusion and Future Work
RPSRNet provides a robust and efficient framework for rigid registration that scales to massive real-world datasets. While highly effective, the authors identify "partial-to-partial" registration—where source and target clouds only partially overlap—as a remaining challenge. Future work will focus on integrating specialized loss functions to improve partial overlap handling and implementing a GPU-based tree builder to further eliminate CPU bottlenecks.
Summary updated