Abstract
Point cloud registration is a critical component in robotics and augmented reality, yet it remains challenging for small objects with partial overlap and sparse geometric features. We present DeepPRO, a novel keypoint-free approach for real-time 3D point cloud registration of previously unseen objects. DeepPRO utilizes a generative neural network to predict the aligned positions of points while preserving their original ordering, thereby establishing dense correspondences without the need for expensive matching algorithms. Our method achieves state-of-the-art accuracy, including a 2.2mm ADD error on the Linemod dataset, while maintaining a high-performance throughput of over 50 FPS on mobile devices.
Introduction
Aligning two 3D scans of the same object—point cloud registration—is fundamental to tasks like 3D reconstruction and robotic manipulation. However, real-world applications involving small objects encounter three significant hurdles. First, partial overlap caused by self-occlusion and sensor noise makes one-to-one matching difficult. Second, feature scarcity in smooth or small objects limits the effectiveness of traditional keypoint-based methods. Finally, there is a constant trade-off between efficiency and accuracy, where classical methods like Iterative Closest Point (ICP) are often too slow for mobile deployment, and many deep learning models fail to generalize beyond synthetic training data.
DeepPRO addresses these challenges by shifting the paradigm from "matching" existing points to "imagining" the aligned state of the object.
Methodology
The core innovation of DeepPRO is its keypoint-free, generative architecture. Unlike traditional methods that search for sparse correspondences between two point clouds ($X_1$ and $X_2$), DeepPRO predicts where the points of $X_2$ would be located if they were already transformed into the coordinate frame of $X_1$.
Architecture Overview
The DeepPRO pipeline is composed of four primary modules designed for end-to-end training and real-time inference:
- Shared Encoder: The model employs a Dynamic Graph CNN (DGCNN) to extract both local ($f^l$) and global ($f^g$) geometric features from the input point clouds. This dual-feature approach ensures the model captures both fine-grained surface details and the overall shape context.
- Conditional Point Cloud Generation: This module takes the concatenated features from both views to generate a predicted aligned point cloud ($P_{21}$). By utilizing $1 \times 1$ convolutions, the network preserves the original point ordering. This architectural choice is significant because it provides "free" dense correspondences: the $i$-th point in the input corresponds directly to the $i$-th point in the output, bypassing the need for computationally expensive matching steps.
- Transform Estimation Network: Instead of relying on analytical SVD layers, which can be unstable during training, DeepPRO uses a multi-layer perceptron (MLP) to directly regress the rotation (represented as a quaternion) and translation parameters.
- Loss Functions: The network is optimized using a combination of rotation/translation error, the Average Distance of Model Points (ADD) loss, and a cyclic loss ($\mathcal{L}_{cyc}$). The cyclic loss ensures consistency by enforcing that the transformation from view $1 \to 2$ is the mathematical inverse of the transformation from $2 \to 1$.
Datasets and Evaluation
To rigorously test generalization and performance, the authors evaluated DeepPRO on three benchmarks:
- Linemod: A standard dataset for real-world 6D pose estimation.
- ModelNet40: A synthetic CAD model dataset used to evaluate performance under large angular baselines.
- PRO1k: A new, large-scale real-world dataset introduced by the authors, featuring 1,000 diverse objects to test generalization to unseen geometries.
Results and Discussion
DeepPRO significantly outperformed 13 baseline methods, including ICP, DCP, and PRNet. On the Linemod dataset, it achieved a 2.2mm ADD error and a rotation error of 1.19°.
Key insights from the results include:
- Robustness to Smooth Surfaces: By focusing on global shape "flow" rather than discrete keypoints, DeepPRO remains accurate even when objects lack sharp geometric features.
- Real-time Mobile Performance: The model achieves over 50 FPS on mobile devices, making it one of the few deep-learning-based registration methods suitable for on-device augmented reality.
- Generalization: DeepPRO maintained high accuracy on the unseen objects of the PRO1k dataset, demonstrating a successful bridge over the "domain gap" that often plagues models trained solely on synthetic data.
Conclusion and Future Work
DeepPRO demonstrates that generative point cloud prediction is a powerful alternative to traditional keypoint matching for real-time registration. While highly effective for frame-to-frame tracking, future work will explore improving performance for very large angular differences. Additionally, the authors plan to incorporate multi-modal data (RGB-D) to resolve ambiguities in symmetric objects and investigate methods to handle transparent or reflective materials that typically challenge depth sensors.
Summary updated