Abstract
Motivated by the stringent requirements of unstructured real-world where a plethora of unknown objects reside in arbitrary locations of the surface, we propose a voxel-based deep 3D Convolutional Neural Network (3D CNN) that generates feasible 6-DoF grasp poses in unrestricted workspace with reachability awareness. Unlike the majority of works that predict if a proposed grasp pose within the restricted workspace will be successful solely based on grasp pose stability, our approach further learns a reachability predictor that evaluates if the grasp pose is reachable or not from robot's own experience. To avoid the laborious real training data collection, we exploit the power of simulation to train our networks on a large-scale synthetic dataset. This work is an early attempt that simultaneously evaluates grasping reachability from learned knowledge while proposing feasible grasp poses with 3D CNN. Experimental results in both simulation and real-world demonstrate that our approach outperforms several other methods and achieves 82.5% grasping success rate on unknown objects.
Introduction
In robotic manipulation, the challenge of grasping in unconstrained, cluttered environments—such as a messy table—requires more than just identifying a stable grip. While traditional research focuses on grasp stability (ensuring the gripper won't drop the object), a stable grasp is practically useless if the robot's arm cannot kinematically reach the target pose due to joint limits or environmental obstacles.
Existing 6-DoF (Degrees of Freedom) grasping methods often rely on computationally expensive motion planning to verify feasibility after a grasp is proposed. This sequential approach often leads to high failure rates and slow execution. This paper introduces a "reachability-aware" framework that bridges this gap by integrating kinematic feasibility directly into the grasp generation process.
Methods
The proposed system decouples the geometric problem of stability from the kinematic problem of reachability using a two-pronged neural architecture.
3D CNN Grasp Predictor
The pipeline begins by taking a point cloud of the scene, performing planar segmentation, and voxelizing a small 3D region ($0.1m^3$) around potential grasp points. A 3D CNN evaluates these voxel grids to predict the probability of a stable grasp ($p_s$), learning to recognize geometric features that facilitate force closure.
Reachability Predictor (RP)
Complementing the stability model is a lightweight Reachability Predictor. This network takes a 6D grasp pose (position and orientation) as input and predicts the probability ($p_r$) that the robot can reach it. By decoupling this from object geometry, the RP focuses solely on the robot's kinematic constraints, acting as a fast learned approximation of a motion planner.
Integrated Grasp Selection
The final grasp success probability is calculated as $p_g = p_s \times p_r$. This allows the robot to prioritize candidates that are both stable and reachable. Furthermore, the use of a fixed-size voxel grid enables an "onion-peeling" strategy, where the robot naturally prioritizes objects on the periphery of a pile, clearing clutter layer by layer.
Datasets and Training
To overcome data scarcity, the authors employed a self-supervised simulation-to-real pipeline. The 3D CNN was trained on 60,000 labeled voxel grids generated from primitive shapes in simulation. The RP was trained on 10,000 self-explored kinematic samples. Real-world evaluation involved 30 toy blocks for clutter and 5 novel household objects (e.g., a power bank, bottles) to test generalization.
Results and Discussion
The approach achieved an 82.5% grasping success rate on novel objects in the real world, outperforming baseline methods like GPD and VPG. Key insights include:
- Efficiency: In simulation, the Reachability Predictor improved the planning success rate from 37.33% to 96%, significantly reducing time spent on impossible moves.
- Robustness: Voxelization proved effective for bridging the sim-to-real gap, as depth data is more consistent than RGB textures.
- Clutter Mastery: The system achieved a 100% completion rate in clearing a workspace of 30 randomly dropped objects.
Conclusion
This work demonstrates that reachability awareness is crucial for efficient 6-DoF grasping in unstructured environments. By learning to approximate kinematic feasibility, the system avoids the computational bottlenecks of traditional motion planning. Future work will explore enhancing robustness to sensor noise and applying the Reachability Predictor to other manipulation tasks like pushing and placing.
Summary updated