GE-Grasp: Efficient Target-Oriented Grasping in Dense Clutter

Zhan Liu, Ziwei Wang, Sichao Huang, Jie Zhou, Jiwen Lu

Introduction

Robotic manipulation in unstructured environments remains a significant challenge, particularly when a robot must retrieve a specific target object from a dense pile of clutter. This "messy room" problem is complicated by two primary factors: occlusion, where the target is partially or fully hidden, and crowdedness, where the lack of physical space prevents a gripper from accessing the target without colliding with surrounding obstacles.

Researchers from Tsinghua University have introduced GE-Grasp, a framework designed to navigate these complexities using a "generator-evaluator" architecture. By intelligently choosing between picking up a target, pushing obstacles, or removing blocking objects entirely, GE-Grasp achieves high success rates with significantly fewer motions than previous state-of-the-art methods.

The GE-Grasp Framework

The core of GE-Grasp is its hierarchical decision-making process, which breaks the manipulation task into three distinct action primitives:

  1. Target-oriented grasping: Directly picking up the goal object.
  2. Pushing: Moving obstacles to create sufficient clearance for the gripper.
  3. Nontarget-oriented grasping: Picking up and removing an obstacle that is physically blocking the target.

Generator-Evaluator Architecture

The system operates through a structured workflow that combines heuristic filtering with deep learning-based evaluation:

  • Perception and Mapping: An RGB-D camera captures a top-down view of the workspace to generate a color image and a depth heightmap. A segmentation module is employed to identify the target's location within the scene.
  • Action Generation and the Spatial Correlation Test (SCT): Rather than evaluating every possible pixel in the workspace, the system uses "Generators" to propose candidate actions. These candidates are filtered through the Spatial Correlation Test (SCT)—a set of heuristic rules that analyze height differences in the map. The SCT ensures that the gripper can physically fit into the proposed space or grasp an object without colliding with adjacent items, effectively pruning the search space without the need for computationally expensive 3D collision checking.
  • Action Evaluation: The remaining candidates are processed by DenseNet-121 neural networks. These evaluators predict a "benefit score" for each action, estimating how much the move will contribute to the eventual successful retrieval of the target.
  • Conditional-Greedy Policy: The robot follows a hierarchy of preferences. If a target-grasp action yields a high enough score, it is executed immediately. If not, the robot selects the highest-scoring auxiliary action (pushing or nontarget grasping) to clear the workspace.

Experimental Evaluation

The authors trained the GE-Grasp models using 8,800 samples collected in the V-REP simulation environment. The framework was then tested across several increasingly difficult scenarios:

  • Random Clutter: Piles consisting of 10, 15, or 20 blocks.
  • Challenging Clutter: Eight hand-designed "adversarial" cases where the target is intentionally obstructed by heavy or tightly packed blocks.
  • Grasping the Invisible: Scenarios where the target is completely buried and must be uncovered through exploratory actions.
  • Real-World Generalization: A physical test using 12 common household items (e.g., toys, chargers, staplers) to evaluate the model's ability to transfer from simulation to reality.

Key Results

GE-Grasp demonstrated superior performance compared to existing baselines such as MASK-VPG and Grasping-Invisible (GI):

  • Success Rate: In random clutter, GE-Grasp achieved a 98.8% success rate, significantly outperforming the 86.7% achieved by the next best method.
  • Motion Efficiency: The system required an average of only 1.38 motions to retrieve the target, compared to nearly 2.0 motions for previous methods.
  • Robustness: In adversarial cases, GE-Grasp maintained a 95% success rate. Notably, when the ability to push was disabled, the success rate dropped to 77.2%, highlighting the importance of action synergy.
  • Real-World Performance: When tested on novel objects in a physical environment, the system maintained a 94.2% success rate, confirming the robustness of the simulation-to-real transfer.

Discussion and Future Work

The success of GE-Grasp stems from its ability to "prepare the battlefield." By recognizing that removing an obstacle is often more efficient than simply pushing it, the system optimizes the workspace for the final grasp. The use of the SCT as a pre-filter allows the system to remain computationally efficient while avoiding obvious physical failures.

However, some limitations remain. The current logic relies on the segmentation module identifying at least a portion of the target to initiate the retrieval process. Additionally, the model assumes a static scene between actions, which may not account for objects that roll or shift unpredictably. Finally, the SCT heuristics are currently tailored for parallel-jaw grippers; adapting the framework for multi-fingered robotic hands would require a redesign of the spatial rules. Future research will likely focus on dynamic scene understanding and broader gripper compatibility.

Summary updated

BibTeX
@inproceedings{liu2022ge,
  author = {Zhan Liu and Ziwei Wang and Sichao Huang and Jie Zhou and Jiwen Lu},
  title = {{GE-Grasp: Efficient Target-Oriented Grasping in Dense Clutter}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2022},
  url = {https://openreview.net/forum?id=f9RVZzNwV4z},
}
Figure