Abstract
Estimating the 6D pose of objects from a single 2.5D depth scan in cluttered environments remains a significant challenge in computer vision and robotics. Traditional methods often struggle with local optima or require clean point clouds. This paper introduces a robust, end-to-end deep learning framework that combines instance segmentation with a pose estimation network enhanced by Optimal Transport. By explicitly modeling outliers and handling self-occlusion, the proposed method achieves state-of-the-art "one-shot" registration accuracy and real-time performance on both synthetic and real-world datasets.
Introduction
3D registration—the process of aligning a 3D model to observed points—is fundamental for tasks ranging from robotic manipulation to autonomous navigation. However, real-world scenarios involve "self-occluded objects in context," where sensors only capture partial surfaces (2.5D) amidst significant background clutter. Existing algorithms like Iterative Closest Point (ICP) are prone to failure in these high-outlier environments. This work addresses the gap between complete 3D CAD models and partial, noisy observations by proposing a pipeline that isolates the target object before performing geometric alignment.
Methodology
The proposed architecture follows a two-stage pipeline designed for efficiency and robustness. The first stage utilizes an adapted YOLACT framework for instance segmentation on depth maps, isolating the target object's points from background context like floors and walls.
The second stage performs pose estimation using a network based on Deep Closest Point (DCP-v2). A critical innovation is the integration of an Optimal Transport layer using the Sinkhorn algorithm. Unlike standard SoftMax, this layer includes an "outlier bin," allowing the network to ignore points that lack valid matches due to imperfect segmentation or self-occlusion. Features are extracted using Dynamic Graph CNN (DGCNN) and refined via a Transformer to capture cross-source relationships. To enhance generalization, the authors implemented an on-the-fly rendering strategy using PyTorch3D, exposing the model to a virtually infinite variety of object poses during training.
Experimental Results
The framework was evaluated on ModelNet40, BlenderProc (cluttered synthetic), and TUD-L (real-world) datasets. The method demonstrated superior accuracy compared to traditional handcrafted feature-based methods like TEASER++ and previous learning-based models.
Key findings include:
- Robustness to Clutter: The system maintained high precision even when background points constituted over 95% of the scene.
- Real-Time Performance: The entire pipeline is highly efficient, with the pose estimation stage requiring between 0.03s and 0.11s, making it suitable for live robotics applications.
- Generalization: Evaluations on the TUD-L dataset proved that the model, trained primarily on synthetic data, generalizes effectively to real-world sensor data.
Conclusion
This work demonstrates that combining learned segmentation with robust geometric matching is essential for practical 3D registration in cluttered scenes. While performance remains tied to the quality of initial segmentation, the inclusion of Optimal Transport significantly mitigates noise and occlusion issues. Future research aims to integrate these stages more tightly and deploy the technology for complex tasks such as non-cooperative satellite rendezvous in space.
Summary updated