Self-supervised Point Set Local Descriptors for Point Cloud Registration

Yijun Yuan, Jiawei Hou, Andreas Nüchter, Sören Schwertfeger

Abstract

In this work, we propose to learn local descriptors for point clouds in a self-supervised manner. In each iteration of the training, the input of the network is merely one unlabeled point cloud. On top of our previous work, that directly solves the transformation between two point sets in one step without correspondences, the proposed method is able to train from one point cloud, by supervising its self-rotation, that we randomly generate. The whole training requires no manual annotation. In several experiments we evaluate the performance of our method on various datasets and compare to other state of the art algorithms. The results show, that our self-supervised learned descriptor achieves equivalent or even better performance than the supervised learned model, while being easier to train and not requiring labeled data.

Introduction

Point Cloud Registration (PCR)—the process of aligning two 3D scans into a single coordinate system—is a cornerstone of robotics, SLAM, and 3D reconstruction. Traditionally, this process relies on "descriptors," mathematical fingerprints of local geometry, to find correspondences between scans. While deep learning has significantly improved these descriptors, most state-of-the-art models are supervised, requiring massive amounts of human-labeled data or ground-truth poses. Labeling 3D correspondences is notoriously difficult and time-consuming. This paper addresses this "data hunger" by proposing a self-supervised framework that learns directly from raw, unlabeled point clouds.

Methodology

The core of the proposed approach is a training pipeline that treats registration as a pretext task, enabling end-to-end learning without manual labels.

Training Pipeline

The system takes a single unlabeled point cloud ($PC_1$) and applies a known, random rotation ($R_{random}$) to create a second version ($PC_2$). As shown in Figure 1, both point clouds pass through a shared Descriptor (DESC) network.

The self-supervised training pipeline, showing data augmentation, feature extraction, and the differentiable CF module.

Descriptor Extraction

The DESC module, based on the 3DFeatNet architecture, maps local clusters of points into 32-dimensional descriptor vectors. The internal structure of this module is detailed in Figure 2.

The internal structure of the DESC module, illustrating the two-part 3DFeatNet architecture.

The CF Module and Loss Function

Instead of standard triplet losses, the authors utilize a Full Connection Form Solution (CF) module. This differentiable layer uses Singular Value Decomposition (SVD) to solve for the transformation ($R, t$) between the two point sets in a single step. Because the entire pipeline is differentiable, the network can calculate the error between the predicted rotation and the ground-truth random rotation ($R_{gt} := R_{random}$). This "rotation error" is backpropagated to update the descriptor network.

Experiments and Results

The model was evaluated on two major outdoor LiDAR benchmarks: the Oxford RobotCar Dataset and the KITTI Dataset.

Performance on Oxford RobotCar

The proposed self-supervised model (using an ISS keypoint detector) achieved a 98.10% success rate with a Relative Rotation Error (RRE) of 1.01°. This matches the performance of the supervised 3DFeatNet while being significantly simpler to train and requiring no labeled data.

Generalization to KITTI

On the unseen KITTI dataset, the model demonstrated robust generalization, achieving a 93.50% success rate and an RRE of 0.510°. It outperformed several handcrafted baselines, such as FPFH and USC.

Training Efficiency

The self-supervised approach converges quickly, with the final model saved at 72,500 iterations, roughly equivalent to 20 epochs of supervised training.

Conclusion

This work demonstrates that self-supervised learning is a viable and efficient alternative to supervised methods for point cloud descriptor learning. By leveraging registration as a pretext task, the model achieves state-of-the-art performance without the need for manual annotations. Future work includes expanding the method to full 3D rotations, indoor datasets, and incorporating multi-modal information like RGB data.

Summary updated

Abstract

In this work, we propose to learn local descriptors for point clouds in a self-supervised manner. In each iteration of the training, the input of the network is merely one unlabeled point cloud. On top of our previous work, that directly solves the transformation between two point sets in one step without correspondences, the proposed method is able to train from one point cloud, by supervising its self-rotation, that we randomly generate. The whole training requires no manual annotation. In several experiments we evaluate the performance of our method on various datasets and compare to other state of the art algorithms. The results show, that our self-supervised learned descriptor achieves equivalent or even better performance than the supervised learned model, while being easier to train and not requiring labeled data.
BibTeX
@inproceedings{yuan2021self,
  author = {Yijun Yuan and Jiawei Hou and Andreas Nüchter and Sören Schwertfeger},
  title = {{Self-supervised Point Set Local Descriptors for Point Cloud Registration}},
  booktitle = {Sensors},
  year = {2021},
  url = {https://openreview.net/forum?id=kAf3N3LqBLe},
}
Figure