SD-Net: Symmetric-Aware Keypoint Prediction and Domain Adaptation for 6D Pose Estimation In Bin-picking Scenarios

Ding-Tao Huang, En-Te Lin, Lipeng Chen, Li-Fu Liu, Long Zeng

Abstract

Despite the success in 6D pose estimation in bin-picking scenarios, existing methods still struggle to produce accurate prediction results for symmetric objects and real-world scenarios. The primary bottlenecks include the ambiguity of keypoints caused by object symmetries and the domain gap between real and synthetic data. To circumvent these problems, we propose SD-Net, a new 6D pose estimation network featuring symmetric-aware keypoint prediction and self-training domain adaptation. SD-Net builds on pointwise keypoint regression and deep Hough voting to perform reliable detection under clutter and occlusion. Specifically, we design a robust 3D keypoint selection strategy considering the symmetry class of objects and equivalent keypoints, facilitating accurate localization even in highly occluded scenes. Additionally, we implement an effective filtering algorithm to dynamically eliminate ambiguity and outlier candidates. For domain adaptation, we employ a student-teacher training scheme using tailored heuristics for 3D geometry pseudo-labeling based on semi-chamfer distance. On the public Sil'eane dataset, SD-Net achieves state-of-the-art results with an average precision of 96%. On the Parametric dataset, SD-Net outperforms existing methods by 8%, demonstrating superior learning and generalization abilities.

Introduction

In robotic bin-picking, a robot must identify the 3D position and orientation (6D pose) of objects piled randomly in a bin. This task is notoriously difficult due to heavy occlusion, clutter, and the "symmetry problem." Many current methods rely on keypoints—specific points on an object used to triangulate its pose. However, if an object is symmetric (such as a cylinder or a bolt), different poses can look identical to a neural network, leading to ambiguous keypoint predictions. Furthermore, most models are trained on clean, synthetic data but struggle when deployed on noisy, real-world point clouds—a challenge known as the domain gap. SD-Net aims to solve both by making the network symmetry-aware and enabling it to learn from unlabeled real-world data.

Methodology

Symmetric-Aware Keypoint Prediction

SD-Net introduces a two-pronged approach to handle the geometric complexities of industrial parts:

  1. Adaptive Selection: Instead of picking arbitrary points, SD-Net selects keypoints based on the object's symmetry class (e.g., revolution, mirror, or finite symmetry). By avoiding points on curved surfaces that lack distinct features and instead picking points along rotation axes or mirror planes, the model reduces geometric ambiguity.
  2. Equivalent Keypoint Sets (EKS): The model is trained to predict all "equivalent" keypoints that result from symmetry. This prevents the network from receiving conflicting gradients during training when multiple valid poses appear identical.
  3. Density-based Filtering: During inference, the model uses a modified DBSCAN algorithm to cluster predicted keypoints. This allows the system to select the most reliable candidates and effectively filter out outliers caused by occlusion or sensor noise.

Self-Training Domain Adaptation

To bridge the gap between simulation and reality, SD-Net utilizes a Teacher-Student framework:

  • Teacher-Student Loop: A "Teacher" model is initially trained on synthetic data. It then predicts poses on unlabeled real-world data.
  • Semi-Chamfer Distance Heuristic: To ensure the Teacher only passes on high-quality knowledge, the authors use a 3D geometry metric called semi-chamfer distance. This metric verifies how well the predicted pose matches the actual real-world point cloud. Only high-confidence predictions are converted into "pseudo-labels" to train the "Student" model, which eventually replaces the Teacher in an iterative refinement process.

Experimental Results

The researchers evaluated SD-Net on several key benchmarks to test its robustness and generalization capabilities.

Performance on Standard Benchmarks

  • Sil'eane Dataset: On this standard bin-picking benchmark containing over 2,600 scenarios, SD-Net achieved an Average Precision (AP) of 96%, a 6% improvement over previous leading methods.
  • Parametric Dataset: This dataset tests both learning ability (L-dataset) and generalization to objects with different geometric parameters (G-dataset). SD-Net outperformed the state-of-the-art by 8%, demonstrating that the model understands underlying geometric principles rather than just memorizing shapes.

Ablation Studies and Real-World Validation

Ablation studies highlighted the necessity of the symmetry-aware design. Removing the Equivalent Keypoint Set (EKS) logic caused performance on symmetric objects, such as bricks, to plummet from 0.75 to 0.08 AP.

In physical tests using a Fanuc industrial robot and an RVC X 3D camera, the system achieved a 100% success rate in 10 grasping trials of complex, stacked parts. This confirms that the model’s reliance on 3D geometric consistency (via the semi-chamfer distance) makes it highly robust for industrial applications where parts are often texture-less or metallic.

Conclusion

SD-Net addresses the core challenges of robotic bin-picking by treating symmetry as a mathematical feature rather than a source of error. By combining symmetry-aware keypoint selection with a robust self-training domain adaptation loop, the framework achieves state-of-the-art performance and successfully bridges the sim-to-real gap. Future work will focus on reducing the computational overhead of the iterative filtering process and extending the framework to handle non-rigid objects in multi-robot environments.

Summary updated

Abstract

Despite the success in 6D pose estimation in bin-picking scenarios, existing methods still struggle to produce accurate prediction results for symmetry objects and real world scenarios. The primary bottlenecks include 1) the ambiguity keypoints caused by object symmetries; 2) the domain gap between real and synthetic data. To circumvent these problem, we propose a new 6D pose estimation network with symmetric-aware keypoint prediction and self-training domain adaptation (SD-Net). SD-Net builds on pointwise keypoint regression and deep hough voting to perform reliable detection keypoint under clutter and occlusion. Specifically, at the keypoint prediction stage, we designe a robust 3D keypoints selection strategy considering the symmetry class of objects and equivalent keypoints, which facilitate locating 3D keypoints even in highly occluded scenes. Additionally, we build an effective filtering algorithm on predicted keypoint to dynamically eliminate multiple ambiguity and outlier keypoint candidates. At the domain adaptation stage, we propose the self-training framework using a student-teacher training scheme. To carefully distinguish reliable predictions, we harnesses a tailored heuristics for 3D geometry pseudo labelling based on semi-chamfer distance. On public Sil'eane dataset, SD-Net achieves state-of-the-art results, obtaining an average precision of 96%. Testing learning and generalization abilities on public Parametric datasets, SD-Net is 8% higher than the state-of-the-art method. The code is available at https://github.com/dingthuang/SD-Net.
BibTeX
@inproceedings{huang2024sd,
  author = {Ding-Tao Huang and En-Te Lin and Lipeng Chen and Li-Fu Liu and Long Zeng},
  title = {{SD-Net: Symmetric-Aware Keypoint Prediction and Domain Adaptation for 6D Pose Estimation In Bin-picking Scenarios}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2024},
  url = {https://openreview.net/forum?id=F3WelBFi7g},
}
Figure