OPG-Policy: Occluded Push-Grasp Policy Learning with Amodal Segmentation

Hao Ding, Yiming Zeng, Zhaoliang Wan, Hui Cheng

In the messy reality of a warehouse bin or a cluttered kitchen table, objects rarely sit in plain sight. For a robot, a target object is often just a sliver of color peeking out from under a pile of obstacles. Traditional robotic grasping policies struggle here because they only "see" what is visible, leading to a series of inefficient, trial-and-error pokes and prods.

A new paper from researchers at Sun Yat-Sen University introduces OPG-Policy, a framework that gives robots a form of "X-ray vision" through amodal segmentation to master goal-oriented grasping in dense clutter.

TL;DR

OPG-Policy improves robotic grasping in cluttered scenes by using amodal segmentation to predict the full shape of partially hidden target objects. By integrating these "complete" object masks into a Deep Q-Learning framework with a dedicated action coordinator, the system achieves higher success rates and significantly better motion efficiency than previous state-of-the-art methods.

What Problem Does This Paper Address?

The core challenge is occlusion. When a target object is partially covered, a robot lacks critical information: Where is the object's center? How large is it? Which way is it oriented?

Most existing "push-grasp" policies rely on visible segments. Because the robot doesn't know the true extent of the target, it often executes suboptimal moves—pushing obstacles into the target or attempting to grasp a corner that isn't actually a stable grip point. This results in "low motion efficiency," where the robot takes far too many actions to retrieve a single item.

Methods

The researchers propose a three-part framework to bridge the gap between partial observation and successful retrieval:

  1. Amodal Segmentation Module: Instead of just identifying the visible pixels, this module predicts the amodal mask—the union of the visible and the hidden parts of the target. They leverage the UOAIS (Unseen Object Amodal Instance Segmentation) method and use the Segment Anything Model (SAM) to generate high-quality training data in simulation.
  2. Deep Q-Network (DQN) with Amodal Input: The system takes three heightmaps as input: Color (RGB), Depth, and the predicted Amodal Mask. These are processed through a DenseNet backbone to generate "Q-maps" for two primitive actions: Pushing and Grasping. Each pixel in these maps represents the expected reward for performing that action at that specific location and orientation.
  3. The Coordinator (MLP): Rather than simply picking the highest value from the Q-maps, a three-layer Multi-Layer Perceptron (MLP) acts as a "brain." It looks at the best Q-values alongside "domain knowledge"—specifically the occlusion rate (how much of the target is hidden) and the occlusion border (where the obstacles are crowding the target). This allows the robot to make a strategic choice: "Should I keep clearing obstacles, or is the target exposed enough to grab?"

The team also employed staged adaptive training, a form of curriculum learning where the number of obstacles gradually increased from 3 to 18 as the agent's performance improved.

Datasets

  • Simulation: The model was trained and tested in a V-REP/PyRep environment using a simulated UR5 arm.
  • YCB Dataset: For real-world generalization tests, they used objects from the YCB (Yale-CMU-Berkeley) Object and Model Set, which provides a standardized variety of everyday items.
  • Custom Synthetic Data: They used SAM to automatically label visible parts of objects in simulation to create a dataset for training their amodal segmentation module.

Key Insights and Ideas

  • Amodal Masks as Prior Knowledge: The "aha moment" here is that the amodal mask acts as a powerful spatial prior. Even if the robot only sees 20% of a cube, the amodal mask tells the DQN where the other 80% likely is, allowing the push policy to clear space more effectively.
  • The Coordinator is Essential: An ablation study revealed that using the amodal mask alone isn't enough. Without the Coordinator (the MLP that weighs occlusion rates), the robot often falls into repetitive, failed grasping loops. The Coordinator provides the "common sense" to switch from pushing to grasping at the optimal moment.
  • Dynamic Reward Thresholding: To train the pushing policy, they used a dynamically adaptive threshold ($T_g$). A push is only rewarded if it significantly increases the expected success of a future grasp, preventing the robot from making "lazy" pushes that don't actually improve the scene.

Key Results

The OPG-Policy was tested against strong baselines like GTI and GE-GRASP in both simulation and the real world:

  • Higher Success Rate: In the most difficult simulated "hard" scenarios (30 objects with heavy occlusion), OPG-Policy achieved a 68% success rate, significantly outperforming GE-GRASP (57%) and GTI (33%).
  • Superior Efficiency: In real-world "Generalization Tests" (using objects the robot had never seen before), OPG-Policy reached a 90% success rate with an average of only 6.65 moves per retrieval.
  • Zero-Shot Transfer: Remarkably, the model was trained entirely in simulation and transferred to a real-world UR10 robot setup without any fine-tuning, demonstrating the robustness of the amodal representation.

Limitations and Future Work

While OPG-Policy excels at retrieving a specific target, the authors note that the current framework is designed for goal-oriented tasks (finding one specific item). Future work could expand this to more complex multi-object sorting or "tidying up" tasks.

Additionally, while amodal segmentation is powerful, it is still a prediction. In cases of extreme clutter where the target is 100% occluded (completely buried), the system would likely need to incorporate an "exploration" phase to find the target before the amodal module can begin its work.

Summary updated

Abstract

Goal-oriented grasping in dense clutter, a fundamental challenge in robotics, demands an adaptive policy to handle occluded target objects and diverse configurations. Previous methods typically learn policies based on partially observable segments of the occluded target to generate motions. However, these policies often struggle to generate optimal motions due to uncertainties regarding the invisible portions of different occluded target objects across various scenes, resulting in low motion efficiency. To this end, we propose OPG-Policy, a novel framework that leverages amodal segmentation to predict occluded portions of the target and develop an adaptive push-grasp policy for cluttered scenarios where the target object is partially observed. Specifically, our approach trains a dedicated amodal segmentation module for diverse target objects to generate amodal masks. These masks and scene observations are mapped to the future rewards of grasp and push motion primitives via deep Q-learning to learn the motion critic. Afterward, the push and grasp motion candidates predicted by the critic, along with the relevant domain knowledge, are fed into the coordinator to generate the optimal motion implemented by the robot. Extensive experiments conducted in both simulated and real-world environments demonstrate the effectiveness of our approach in generating motion sequences for retrieving occluded targets, outperforming other baseline methods in success rate and motion efficiency.
BibTeX
@inproceedings{ding2024opg,
  author = {Hao Ding and Yiming Zeng and Zhaoliang Wan and Hui Cheng},
  title = {{OPG-Policy: Occluded Push-Grasp Policy Learning with Amodal Segmentation}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2024},
  url = {https://openreview.net/forum?id=allHIuP4zm},
}
Figure