UniGrasp: Learning a Unified Model to Grasp With Multifingered Robotic Hands

Lin Shao, Fábio Ferreira, Mikael Jorda, Varun Nambiar, Jianlan Luo, Eugen Solowjow, Juan Aparicio Ojea, Oussama Khatib, Jeannette Bohg

Abstract

Traditional robotic grasping models are often constrained to specific hardware configurations, requiring extensive retraining when transitioning between different robotic hands. This paper presents UniGrasp, a unified grasp synthesis framework that generalizes across both novel objects and diverse multi-fingered robotic hands. By utilizing a gripper’s kinematic description (URDF) as input alongside object point clouds, UniGrasp predicts optimal contact points without hand-specific training. Our approach employs a Point Set Selection Network (PSSN) and a ranking-based learning strategy to achieve high success rates across various grippers, including zero-shot generalization to unseen dexterous hands.

Introduction

The ability to manipulate objects is a cornerstone of robotic autonomy. However, the prevailing paradigm in learning-to-grasp involves training models for specific end-effectors. When a robotic platform is upgraded from a simple parallel-jaw gripper to a complex three-fingered dexterous hand, researchers typically must restart the data collection and training process from scratch. This lack of transferability stems from the fact that grasping is a relational problem: a "good" grasp is defined by the intersection of an object's geometry and a robot's specific kinematic constraints.

UniGrasp addresses this challenge by treating the robotic hand as an input variable rather than a fixed constant. By shifting the focus from end-effector pose prediction to contact point selection, UniGrasp provides a universal language for grasping that bridges the gap between disparate robotic hardwares.

Methodology: The Point Set Selection Network (PSSN)

The core of the UniGrasp framework is the Point Set Selection Network (PSSN), which identifies optimal contact locations on an object's surface based on the specific capabilities of the input gripper.

Unified Gripper Representation

To enable hand-agnosticism, the model must "understand" the geometry and reachability of the gripper it is currently controlling. UniGrasp achieves this by processing the gripper’s Unified Robot Description Format (URDF). The authors use an unsupervised autoencoder to map gripper configurations into a latent space. Specifically, the model is provided with nine reference configurations—representing the boundary and central joint limits—allowing the network to internalize the gripper's workspace and physical dimensions.

Multi-Stage Contact Selection

Grasping with multiple fingers is a sequential dependency problem; the placement of the second finger is constrained by the position of the first. PSSN utilizes a multi-stage selection process, akin to a beam search, to manage this complexity:

  1. Stage 1: The network identifies a set of candidate points for the first finger.
  2. Subsequent Stages: Additional contact points are selected conditioned on the previously chosen points and the gripper's kinematic limits. This approach ensures that the resulting set of points is not only geometrically sound for the object but also physically reachable for the robot.

Learning-to-Rank

Rather than treating grasp synthesis as a binary classification (success vs. failure), UniGrasp employs a ranking loss (ListNet). This allows the model to prioritize the most robust contact sets among millions of potential combinations, leading to more reliable performance in real-world scenarios where sensor noise and execution errors are prevalent.

Data Generation and Training

Training a universal model requires a diverse and massive dataset. The researchers generated a synthetic dataset comprising:

  • Objects: 3,275 unique instances derived from 1,000 3D models within the Bullet physics engine.
  • Grippers: 12 distinct robotic hands, ranging from common two-fingered grippers (Franka Emika, Sawyer) to complex three-fingered hands (Robotiq-3F, Barrett).
  • Ground Truth: Labels were generated using FastGrasp to calculate force closure and inverse kinematics (IK), ensuring that the training data only included reachable, collision-free, and stable grasps.

Experimental Results

UniGrasp was evaluated in both simulated environments and on physical hardware, demonstrating significant improvements over traditional baselines.

Simulation Performance

In simulation, the model achieved a success rate of over 90% for its Top-10 predictions when using grippers included in the training set. More importantly, the model showed remarkable zero-shot generalization to hands it had never encountered during training:

  • Unseen Two-Fingered Gripper: 93% success rate.
  • Allegro Hand: 90% success rate.
  • Schunk SVH (Five-fingered): 83% success rate (using a 3-finger strategy).

Comparative Analysis

When compared against the non-learning FastGrasp baseline, UniGrasp showed a substantial performance leap. On the Allegro hand, FastGrasp achieved only a 40% success rate, while UniGrasp’s learning-based approach reached 90%, highlighting the efficiency of the PSSN architecture in navigating complex kinematic constraints.

Discussion and Limitations

UniGrasp represents a significant step toward universal robotic manipulation. By abstracting the grasping problem into contact point selection, the model moves away from the mechanical specifics of joints (prismatic vs. revolute) and focuses on the physics of force closure.

However, several challenges remain. The current implementation relies on geometric heuristics to prune the search space of contact triplets to avoid a combinatorial explosion. Additionally, the model assumes a static environment and relies on a single depth image. Future iterations could incorporate dynamic grasping capabilities and tactile feedback loops to refine grasps in real-time after initial contact is made.

Conclusion

UniGrasp demonstrates that a single, unified model can effectively synthesize grasps for a wide variety of robotic hands and objects. By integrating the robot's own kinematic description into the learning process, UniGrasp paves the way for more flexible and adaptable robotic systems in both industrial and domestic settings.

Summary updated

Abstract

To achieve a successful grasp, gripper attributes such as its geometry and kinematics play a role as important as the object geometry. The majority of previous work has focused on developing grasp methods that generalize over novel object geometry but are specific to a certain robot hand. We propose UniGrasp, an efficient data-driven grasp synthesis method that considers both the object geometry and gripper attributes as inputs. UniGrasp is based on a novel deep neural network architecture that selects sets of contact points from the input point cloud of the object. The proposed model is trained on a large dataset to produce contact points that are in force closure and reachable by the robot hand. By using contact points as output, we can transfer between a diverse set of multifingered robotic hands. Our model produces over 90% valid contact points in Top10 predictions in simulation and more than 90% successful grasps in real world experiments for various known two-fingered and three-fingered grippers. Our model also achieves 93%, 83% and 90% successful grasps in real world experiments for an unseen two-fingered gripper and two unseen multi-fingered anthropomorphic robotic hands.
BibTeX
@article{shao2020unigrasp,
  author = {Lin Shao and Fábio Ferreira and Mikael Jorda and Varun Nambiar and Jianlan Luo and Eugen Solowjow and Juan Aparicio Ojea and Oussama Khatib and Jeannette Bohg},
  title = {{UniGrasp: Learning a Unified Model to Grasp With Multifingered Robotic Hands}},
  journal = {IEEE Robotics Autom. Lett.},
  year = {2020},
  doi = {10.1109/lra.2020.2969946},
}
Figure