Robust 6D Object Pose Estimation in Cluttered Scenes Using Semantic Segmentation and Pose Regression Networks

Arul Selvam Periyasamy, Max Schwarz, Sven Behnke

Introduction

In robotic manipulation, particularly for warehouse automation tasks like bin-picking, precise 6D object pose estimation is a critical requirement. However, real-world environments present significant challenges: heavy clutter and occlusions, the unavailability of high-fidelity 3D CAD models for diverse inventories, and the need for rapid adaptation to new products. This paper introduces a streamlined pipeline that addresses these hurdles by combining semantic segmentation with a specialized pose regression network, enabling the "enrollment" of new items in under 30 minutes without requiring pre-existing 3D models.

Methodology

The proposed system utilizes a multi-stage approach to derive 6D poses from raw RGB-D data.

Overview of the 6D pose estimation pipeline, from input image to estimated poses.
  1. Semantic Segmentation: A RefineNet architecture performs pixel-wise labeling to isolate target objects from background clutter.
  2. Focus Mechanism: The system crops the image around the detected object. To minimize background interference, a "red-channel encoding" is applied, where non-target pixels are pushed toward the red spectrum, helping the regression network ignore noise.
  3. Pose Regression: A Convolutional Neural Network (CNN) regresses a 5D pose, consisting of a 2D translation $(x, y)$ in the image plane and a 3D orientation represented as a unit quaternion $(q_x, q_y, q_z, q_w)$.
  4. Depth Integration: The final 6D pose is completed by projecting the 2D translation into 3D space using depth information from the RGB-D sensor.
  5. Symmetry Handling: To improve training efficiency, the authors explicitly handle object symmetries (e.g., for dumbbells), assigning consistent ground-truth poses to symmetrical orientations.

Data Acquisition and Synthesis

A key innovation is the rapid automated data acquisition process. Instead of relying on CAD models, the system uses an automated turntable to capture 60 views of a new object in minutes.

The synthetic data generation process: from turntable capture to background insertion for cluttered scene simulation.

These isolated object segments are then synthetically "pasted" onto complex backgrounds with varying rotations and occlusions. This process generates a massive, diverse training dataset that simulates the clutter found in actual warehouse bins, allowing the model to learn robust features quickly.

Results and Discussion

The system was evaluated using objects from the 2017 Amazon Robotics Challenge and real-world cluttered scenes. Key findings include:

  • Accuracy: On synthetic validation sets, the model achieved a translation error of approximately 9.7 pixels and an orientation error of 7.4°.
  • Refinement: In real-world tests, the predicted poses served as excellent initializations for Iterative Closest Point (ICP) refinement, which further reduced orientation errors (e.g., from 14.6° to 14.2° for a brush).
  • Generalization: The network demonstrated an ability to generalize to unseen instances within a category, estimating the pose of a novel power drill with an orientation error of ~33°.
  • Efficiency: By accounting for symmetry, the model converged three times faster (100 vs 300 epochs) with higher final accuracy.

Conclusion

This research presents a practical, high-speed solution for 6D pose estimation in dynamic robotic environments. By bypassing the need for 3D models and leveraging synthetic data generation, the system allows for rapid deployment to new inventory. While challenges remain regarding deformable objects and segmentation dependencies, the pipeline's efficiency and robustness make it highly suitable for real-time robotic applications.

Summary updated

BibTeX
@inproceedings{periyasamy2018robust,
  author = {Arul Selvam Periyasamy and Max Schwarz and Sven Behnke},
  title = {{Robust 6D Object Pose Estimation in Cluttered Scenes Using Semantic Segmentation and Pose Regression Networks}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2018},
  url = {https://openreview.net/forum?id=L5Qyd3s0VB_Y},
}
Figure