Learning to Generate 6-DoF Grasp Poses with Reachability Awareness

Xibai Lou, Yang Yang, Changhyun Choi

TL;DR

This paper introduces a "reachability-aware" grasping framework that combines a 3D Convolutional Neural Network (3D CNN) for predicting grasp stability with a dedicated Reachability Predictor (RP). By decoupling the geometric problem of "will this slip?" from the kinematic problem of "can the arm get there?", the system achieves an 82.5% success rate on novel objects in cluttered, unrestricted real-world environments.

What Problem Does This Paper Address?

In robotic manipulation, most research focuses on grasp stability—analyzing the geometry of an object to find a pose where the gripper won't drop it. However, in unconstrained real-world settings (like a messy table), a "stable" grasp is useless if the robot's arm can't actually reach that position due to joint limits or collisions with other objects.

Traditional 6-DoF (Degrees of Freedom) grasping often relies on computationally expensive motion planning to check feasibility after a grasp is proposed. This leads to high failure rates and slow execution in dense clutter. The authors aim to bridge this gap by making the grasp generation process "reachability-aware" from the start, allowing the robot to prioritize candidates that are both stable and kinematically feasible.

Methods

The authors propose a two-pronged approach integrated into a single pipeline:

  1. 3D CNN Grasp Predictor: The system takes a point cloud of the scene, segments it, and voxelizes a small 3D region around a potential grasp point. A 3D CNN then evaluates this voxel grid to predict the probability of a stable grasp ($p_s$). This model is trained to recognize geometric features that lead to successful force closure.
  2. Reachability Predictor (RP): This is a lightweight neural network that takes a 6D grasp pose (position and orientation) as input and predicts whether the robot can reach it ($p_r$). Crucially, this is decoupled from the object's shape, focusing entirely on the robot's kinematic constraints.
  3. The "Onion-Peeling" Strategy: The final grasp success probability is calculated as $p_g = p_s \times p_r$. By cropping the voxel grid to a specific size ($0.1m^3$), the robot naturally prioritizes objects on the periphery of a pile—effectively "peeling" the clutter layer by layer.
  4. Sampling: Instead of restricted top-down movements, the algorithm uniformly samples grasp candidates across a 6-DoF space, allowing for side-on or angled grasps that are often necessary in complex environments.

Datasets

To overcome the "data hunger" of deep learning, the authors used a self-supervised simulation-to-real pipeline:

  • Synthetic Training: They used 8 primitive shapes to generate a large-scale dataset in simulation (V-REP/Bullet).
  • Scale: The 3D CNN was trained on 60,000 labeled voxel grids, while the RP was trained on 10,000 self-explored kinematic samples.
  • Real-World Evaluation: The system was tested on 30 toy blocks (for clutter) and 5 novel household objects (a banana, power bank, Advil bottle, tape, and Perrier bottle) to prove generalization.

Key Insights and Ideas

  • Decoupling is Key: By separating stability (geometry-dependent) from reachability (robot-dependent), the RP becomes a modular component. You could theoretically swap the robot arm, retrain only the lightweight RP, and keep the heavy 3D CNN intact.
  • Reachability as a Filter: The RP acts as a learned approximation of a motion planner. It filters out impossible poses instantly, avoiding the "computationally expensive" trap of running full inverse kinematics for every candidate.
  • Voxelization for Generalization: Using 3D voxels instead of 2D images makes the model more robust to "sim-to-real" gaps, as depth data is more consistent across environments than RGB textures.

Key Results

  • High Success Rate: The approach achieved an 82.5% grasping success rate on novel objects in the real world, outperforming baseline methods like GPD (Grasp Pose Detection) and VPG (Visual Pushing for Grasping).
  • Efficiency Gains: In simulation ablation studies, adding the Reachability Predictor improved the planning success rate from 37.33% to 96%. This means the robot spent far less time attempting impossible moves.
  • Clutter Mastery: In a "challenging" scenario with 30 randomly dropped objects, the system achieved a 100% completion rate, successfully clearing the workspace.

Limitations and Future Work

  • Sensor Noise: The sampling algorithm relies on point clouds, which can be noisy. The authors note that the system sometimes proposes "vacant" grasp points if the sensor data is poor.
  • Voxel Grid Constraints: The fixed size of the voxel grid ($0.1m$) means the robot prefers smaller objects or specific parts of larger objects (like the neck of a bottle). It may struggle with very large, uniform surfaces.
  • Future Directions: The authors suggest investigating voxel-based methods to enhance robustness to noise and exploring whether the Reachability Predictor can be applied to other tasks like pushing or placing.

Summary updated

Abstract

Motivated by the stringent requirements of unstructured real-world where a plethora of unknown objects reside in arbitrary locations of the surface, we propose a voxel-based deep 3D Convolutional Neural Network (3D CNN) that generates feasible 6-DoF grasp poses in unrestricted workspace with reachability awareness. Unlike the majority of works that predict if a proposed grasp pose within the restricted workspace will be successful solely based on grasp pose stability, our approach further learns a reachability predictor that evaluates if the grasp pose is reachable or not from robot's own experience. To avoid the laborious real training data collection, we exploit the power of simulation to train our networks on a large-scale synthetic dataset. This work is an early attempt that simultaneously learns grasping reachability while proposing feasible grasp poses with 3D CNN. Experimental results in both simulation and real-world demonstrate that our approach outperforms several other methods and achieves 82.5% grasping success rate on unknown objects.
BibTeX
@inproceedings{lou2020learning,
  author = {Xibai Lou and Yang Yang and Changhyun Choi},
  title = {{Learning to Generate 6-DoF Grasp Poses with Reachability Awareness}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2020},
  url = {https://openreview.net/forum?id=uz15QkKuG6},
}

Loading related content...

Figure