RPSRNet: End-to-End Trainable Rigid Point Set Registration Network using Barnes-Hut 2D-Tree Representation

Sk Aziz Ali, Kerem Kahraman, Gerd Reis, Didier Stricker

Abstract

We propose RPSRNet - a novel end-to-end trainable deep neural network for rigid point set registration. For this task, we use a novel $2^D$-tree representation for the input point sets and a hierarchical deep feature embedding in the neural network. An iterative transformation refinement module in our network boosts the feature matching accuracy in the intermediate stages. We achieve an inference speed of 12-15ms to register a pair of input point clouds as large as 250K. Extensive evaluation on (i) KITTI LiDAR odometry and (ii) ModelNet-40 datasets shows that our method outperforms prior state-of-the-art methods - e.g., on the KITTI data set, DCP-v2 by 1.3 and 1.5 times, and PointNetLK by 1.8 and 1.9 times better rotational and translational accuracy respectively. Evaluation on ModelNet40 shows that RPSRNet is more robust than other benchmark methods when the samples contain a significant amount of noise and other disturbances. RPSRNet accurately registers point clouds with non-uniform sampling densities, e.g., LiDAR data, which cannot be processed by many existing deep-learning-based registration methods.

Introduction

Rigid Point Set Registration (RPSR) is the fundamental task of aligning a "source" point cloud to a "target" point cloud by finding the optimal rotation and translation. This process is critical for applications in robotics, autonomous driving, and 3D reconstruction. Traditional methods like Iterative Closest Point (ICP) often suffer from local minima, while modern deep learning approaches face challenges with memory consumption (voxel-based) or capturing local spatial correlations (point-based).

RPSRNet addresses these limitations by introducing a Barnes-Hut $2^D$-tree representation. This structure allows the network to be memory-efficient while remaining spatially aware, effectively bridging the gap between sparse point clouds and dense volumetric representations.

Overview of the RPSRNet pipeline, illustrating the registration process from initial source and target point clouds to the final registered output using the Barnes-Hut tree representation.

Proposed Method: Barnes-Hut Tree and Hierarchical Encoding

The core of RPSRNet is the use of a Barnes-Hut (BH) tree representation. For 3D data, this manifests as an octree where the input space is recursively subdivided.

BH-Tree Representation

Each non-empty node in the tree stores the Center-of-Mass (CoM) of the points within its volume and an Inverse Density (ID) value. This representation allows the network to ignore empty space, significantly reducing computational overhead compared to dense voxel grids. The use of Morton "Z" curves for indexing tree nodes further accelerates neighbor lookups during convolution.

Hierarchical Feature Encoder (HFE)

The network employs a Hierarchical Feature Encoder that performs "tree-convolutions." By fetching 26 adjacent nodes at each depth, the network learns geometric features at multiple scales, from coarse global structures to fine local details. The architecture uses late fusion to combine positional features (HPFE) and density features (HDFE), making the model robust to the non-uniform sampling densities typical of LiDAR sensors.

The architecture of RPSRNet at iteration $k$, showing the dual-stream Hierarchical Feature Encoder (HFE), the Transformer network for feature matching, and the differentiable SVD layer for transformation regression.

Iterative Refinement and Regression

RPSRNet utilizes a Transformer block to learn contextual residuals between source and target embeddings. A differentiable Singular Value Decomposition (SVD) layer then regresses the final transformation parameters ($R, t$). The process is iterative: the network predicts a transformation, applies it to the source, and repeats the process to refine the alignment, which significantly boosts final accuracy.

Experimental Evaluation

The authors evaluated RPSRNet on two major benchmarks: the synthetic ModelNet40 dataset and the real-world KITTI LiDAR-odometry dataset.

Performance and Robustness

  • Speed: RPSRNet achieves an inference time of 12–15 ms for point clouds with up to 250k points, making it suitable for real-time applications.
  • KITTI Results: RPSRNet demonstrated superior performance over state-of-the-art methods like DCP-v2 and PointNetLK, achieving up to 1.9x better translational accuracy.
  • Noise Robustness: On ModelNet40, RPSRNet remained stable even with 40% noise levels, whereas competing methods showed significant performance degradation.
  • Density Adaptivity: The network successfully registered LiDAR frames including ground points, a scenario where many unsupervised or classical methods fail due to density variations.

Conclusion

RPSRNet introduces a powerful framework for rigid point set registration by leveraging the efficiency of Barnes-Hut trees and the representational power of hierarchical deep embeddings. Its ability to handle massive, non-uniformly sampled point clouds in real-time makes it a significant advancement for autonomous systems. Future work includes extending the model to handle partial-to-partial registration and non-rigid scene flow estimation.

Summary updated

Abstract

We propose RPSRNet - a novel end-to-end trainable deep neural network for rigid point set registration. For this task, we use a novel $2^D$-tree representation for the input point sets and a hierarchical deep feature embedding in the neural network. An iterative transformation refinement module in our network boosts the feature matching accuracy in the intermediate stages. We achieve an inference speed of 12-15ms to register a pair of input point clouds as large as 250K. Extensive evaluation on (i) KITTI LiDAR odometry and (ii) ModelNet-40 datasets shows that our method outperforms prior state-of-the-art methods - e.g., on the KITTI data set, DCP-v2 by1.3 and 1.5 times, and PointNetLK by 1.8 and 1.9 times better rotational and translational accuracy respectively. Evaluation on ModelNet40 shows that RPSRNet is more robust than other benchmark methods when the samples contain a significant amount of noise and other disturbances. RPSRNet accurately registers point clouds with non-uniform sampling densities, e.g., LiDAR data, which cannot be processed by many existing deep-learning-based registration methods.
BibTeX
@inproceedings{ali2021rpsrnet,
  author = {Sk Aziz Ali and Kerem Kahraman and Gerd Reis and Didier Stricker},
  title = {{RPSRNet: End-to-End Trainable Rigid Point Set Registration Network using Barnes-Hut 2D-Tree Representation}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2021},
  url = {https://openreview.net/forum?id=E7PzEuzuONE},
}
Figure