Center-Based Decoupled Point Cloud Registration for 6D Object Pose Estimation

Haobo Jiang, Zheng Dang, Shuo Gu, Jin Xie, Mathieu Salzmann, Jian Yang

TL;DR

This paper introduces a novel "center-based decoupled" approach to 6D object pose estimation from point clouds. By first predicting the object's center and decoupling translation from rotation, their method achieves significantly improved robustness in challenging real-world scenarios, outperforming existing state-of-the-art methods.

What Problem Does This Paper Address?

Estimating an object's 6D pose (3D position and 3D orientation) from point cloud data is a critical task for applications like robotics, augmented reality, and autonomous navigation. Traditional methods often struggle with real-world data due to factors like partial scans, noise, and occlusions. Many existing deep learning approaches, while powerful, still face challenges in achieving high accuracy and robustness on these complex datasets. A key difficulty lies in the intertwined nature of rotation and translation estimation, where errors in one can propagate to the other. This paper aims to tackle this by decoupling these two components, specifically by focusing on the object's center.

Methods

The core innovation is a "center-based decoupled registration" framework. Here's a breakdown of the key components:

  • Decoupling Translation via Center Prediction: Instead of directly regressing the full transformation, the method first predicts the 3D center of the source point cloud. Since the model point cloud is assumed to be centered at the origin, the translation vector is simply the negative of the predicted source center. This is a "correspondence-free" approach to translation, which significantly reduces the difficulty and improves robustness.
  • Multi-view Center Predictor: To accurately predict the object center, the system uses a multi-view approach. It extracts features from both a bird's-eye view (BEV) and a front view (FV) of the source point cloud. These views jointly "vote" for the object's center.
  • Model Shape Embedding (MOSE): To address the issue of missing shape information in partially scanned objects, a MOSE module is introduced. This module augments the source point cloud features by retrieving relevant shape cues from the complete model point cloud, effectively "completing" the missing information.
  • Center-Aware Rotation Estimation: Once the translation is estimated and the point clouds are center-aligned, the rotation is predicted. This part leverages feature similarity to establish correspondences for SVD-based rotation estimation.
  • Center-Aware Hybrid Feature Descriptor: To improve the quality of correspondences, a novel feature descriptor is proposed. This descriptor incorporates "part-aware" information by encoding a point's relative position to the object's center, helping to distinguish locally similar but non-corresponding points.
  • Normal Correction: To further enhance feature consistency, a center-based normal orientation correction technique is employed. This ensures that normal vectors point consistently, mitigating orientation ambiguities.

Datasets

The paper evaluates its method on three widely-used real-world datasets:

  • TUD-L: Contains household objects under varying lighting conditions.
  • LINEMOD: Features texture-less objects in cluttered scenes.
  • Occluded-LINEMOD: A subset of LINEMOD with significant occlusions.

The authors follow standard data splits for training and testing as defined in previous works.

Key Insights and Ideas

  • Decoupling is Key for Robustness: The most significant insight is that decoupling translation estimation by focusing on the object's center dramatically improves robustness, especially in challenging real-world scenarios where establishing correspondences is difficult.
  • Leveraging Model Shape for Partial Scans: The Model Shape Embedding (MOSE) module is a clever way to compensate for missing data in source point clouds by intelligently borrowing information from the complete model. This is crucial for accurate center prediction.
  • Part-Aware Features for Better Correspondences: The center-aware hybrid feature descriptor, by highlighting a point's position relative to the object's center, provides a more discriminative representation that helps overcome ambiguities in local geometry.
  • Normal Correction for Orientation Consistency: Addressing the inherent ambiguity in surface normals is vital for reliable feature matching, and the proposed center-based normal correction is an effective way to achieve this.

Key Results

The proposed "CenterReg" method consistently outperforms state-of-the-art methods across all tested datasets and metrics.

  • TUD-L: Achieves the best performance, with significant improvements in mAP@5° (10%↑) and mAP@1cm (9%↑).
  • LINEMOD: Demonstrates substantial precision gains, up to 20% in mAP@10° and 56% in mAP@1cm.
  • Occluded-LINEMOD: Despite severe occlusions, the method still achieves top performance on most criteria, with particularly strong translation prediction due to the MOSE module.
  • ICP Refinement: Adding ICP-based pose refinement further boosts rotation accuracy, though translation precision can slightly decrease, suggesting the robustness of their learned translation regression.

Limitations and Future Work

  • Limitations: The paper notes that while their method significantly improves robustness, it still relies on a complete model point cloud. The performance might degrade if the model itself is significantly incomplete or inaccurate. The use of softmax in the MOSE module is highlighted as a simple but effective choice, implying that more sophisticated attention mechanisms could potentially offer further improvements.
  • Future Work: The authors suggest exploring more advanced feature descriptors and potentially extending the framework to handle more complex scenarios, such as non-rigid object registration or scenes with multiple objects. The paper also mentions that further discussion on limitations and future work can be found in Appendix C, which is not included in the provided text.

Summary updated

Abstract

In this paper, we propose a novel center-based decoupled point cloud registration framework for robust 6D object pose estimation in real-world scenarios. Our method decouples the translation from the entire transformation by predicting the object center and estimating the rotation in a center-aware manner. This center offset-based translation estimation is correspondence-free, freeing us from the difficulty of constructing correspondences in challenging scenarios, thus improving robustness. To obtain reliable center predictions, we use a multi-view (bird’s eye view and front view) object shape description of the source-point features, with both views jointly voting for the object center. Additionally, we propose an effective shape embedding module to augment the source features, largely completing the missing shape information due to partial scanning, thus facilitating the center prediction. With the center-aligned source and model point clouds, the rotation predictor utilizes feature similarity to establish putative correspondences for SVD-based rotation estimation. In particular, we introduce a center-aware hybrid feature descriptor with a normal correction technique to extract discriminative, part-aware features for high-quality correspondence construction. Our experiments show that our method outperforms the state-of-the-art methods by a large margin on real-world datasets such as TUD-L, LINEMOD, and Occluded-LINEMOD. Code is available at https://github.com/JiangHB/CenterReg.
BibTeX
@inproceedings{jiang2023center,
  author = {Haobo Jiang and Zheng Dang and Shuo Gu and Jin Xie and Mathieu Salzmann and Jian Yang},
  title = {{Center-Based Decoupled Point Cloud Registration for 6D Object Pose Estimation}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2023},
  doi = {10.1109/iccv51070.2023.00317},
}

Loading related content...

Figure