Abstract: Learning robotic grasps from visual observations is a promising yet challenging task. Recent research shows its great potential by preparing and learning from large-scale synthetic datasets. For the popular, 6 degree-of-freedom (6-DOF) grasp setting of parallel-jaw gripper, most of existing methods take the strategy of heuristically sampling grasp candidates and then evaluating them using learned scoring functions. This strategy is limited in terms of the conflict between sampling efficiency and coverage of optimal grasps. To this end, we propose in this work a novel, end-to-end Grasp Proposal Network (GPNet), to predict a diverse set of 6-DOF grasps for an unseen object observed from a single and unknown camera view. GPNet builds on a key design of grasp proposal module that defines anchors of grasp centers at discrete but regular 3D grid corners, which is flexible to support either more precise or more diverse grasp predictions. To test GPNet, we contribute a synthetic dataset of 6-DOF object grasps; evaluation is conducted using rule-based criteria, simulation test, and real test. Comparative results show the advantage of our methods over existing ones. Notably, GPNet gains better simulation results via the specified coverage, which helps achieve a ready translation in real test.
1. Introduction
Determining a 6-degree-of-freedom (6-DOF) grasp pose from a single, partial camera view is a fundamental challenge in robotics. Traditional approaches often rely on heuristic sampling of hundreds of candidates followed by neural network scoring. This process is computationally expensive and often fails to cover optimal grasps. Generative models, while faster, frequently suffer from "mode collapse," where predicted grasps cluster around a single point (like the center of mass), lacking the diversity required for complex environments.
We introduce the Grasp Proposal Network (GPNet), an end-to-end solution that leverages the concept of "anchors" from 2D object detection to generate diverse and accurate 6-DOF grasps. By defining potential grasp centers on a regular 3D grid, GPNet ensures broad coverage of the object's surface and geometry.
2. Grasp Proposal Network Architecture
The GPNet architecture consists of three primary stages: point-wise feature extraction, a grasp proposal module, and multi-headed output prediction.
The network first processes the input point cloud through PointNet++ to obtain point-wise features. The Grasp Proposal Module then defines a 3D grid of anchors. Observed surface points are paired with these anchors to form initial proposals. To manage the proposal space, geometric pruning is applied to remove physically improbable candidates.
A key innovation is the use of anchor-dependent features. Instead of global context, the network extracts features based on the relative geometry between surface points and the specific grid anchor. This localized context significantly improves the accuracy of the subsequent regression and classification heads:
- Antipodal Classifier: Determines the physical validity of the grasp (ensuring it won't slip).
- Grasp Regressor: Predicts fine-grained offsets for the 3D center and orientation.
- Grasp Classifier: Provides a final confidence score for the proposed grasp.
3. Anchor-Driven Diversity
The anchor-based approach inherently promotes grasp diversity. By distributing potential centers across a rigid grid, the network is forced to explore the entire spatial volume of the object. This prevents the model from converging on a single "best" grasp and provides the robot with multiple options, which is critical when certain approach angles are obstructed by kinematics or obstacles.
4. Experimental Results
GPNet was trained on a large-scale synthetic dataset containing 22.6 million annotated grasps across 226 CAD models. The dataset labels are derived from physics-based simulations in PyBullet, accounting for friction, gravity, and collisions.
In simulation, GPNet achieved a 90% success rate for the top 10% of predictions, significantly outperforming the 6-DOF GraspNet baseline (80% with refinement and only 43.3% without). Crucially, GPNet's coverage rate reached 30.8%, nearly double that of the baseline (16.8%). Real-world experiments on a UR5 robot arm confirmed these findings, with GPNet achieving an 85% success rate on novel objects, demonstrating robust generalization from synthetic training data.
5. Conclusion
GPNet offers a highly efficient, end-to-end framework for robotic grasping. By integrating 3D grid anchors, the model achieves a superior balance of accuracy and diversity compared to heuristic sampling or purely generative methods. Future work will focus on extending this approach to cluttered environments and incorporating real-time physical property inference directly from visual or tactile feedback.
Summary updated