UniGrasp: Learning a Unified Model to Grasp with N-Fingered Robotic Hands

Lin Shao, Fábio Ferreira, Mikael Jorda, Varun Nambiar, Jianlan Luo, Eugen Solowjow, Juan Aparicio Ojea, Oussama Khatib, Jeannette Bohg

Abstract

To achieve a successful grasp, gripper attributes such as its geometry and kinematics play a role as important as the object geometry. The majority of previous work has focused on developing grasp methods that generalize over novel object geometry but are specific to a certain robot hand. We propose UniGrasp, an efficient data-driven grasp synthesis method that considers both the object geometry and gripper attributes as inputs. UniGrasp is based on a novel deep neural network architecture that selects sets of contact points from the input point cloud of the object. The proposed model is trained on a large dataset to produce contact points that are in force closure and reachable by the robot hand. By using contact points as output, we can transfer between a diverse set of multifingered robotic hands. Our model produces over 90% valid contact points in Top10 predictions in simulation and more than 90% successful grasps in real world experiments for various known two-fingered and three-fingered grippers. Our model also achieves 93%, 83% and 90% successful grasps in real world experiments for an unseen two-fingered gripper and two unseen multi-fingered anthropomorphic robotic hands.

Introduction

In robotic manipulation, hardware generalization remains a significant challenge. Most existing grasp synthesis models are designed for specific end-effectors; a model trained for a parallel-jaw gripper cannot be easily adapted for a multi-fingered anthropomorphic hand. This hardware-specific nature necessitates retraining models from scratch whenever the robotic platform changes.

UniGrasp addresses this bottleneck by shifting the focus from robot-centric parameters (like 6D wrist poses) to object-centric contact points. By predicting where the fingers should touch the object to ensure stability and reachability, UniGrasp creates a universal interface that works across diverse robotic hands.

UniGrasp utilizes contact points as a universal representation to enable grasping across various objects and gripper types.

Methodology

The UniGrasp framework consists of two primary components: a hardware-aware gripper embedding and a multi-stage point selection network.

Gripper Embedding

To incorporate the robot's physical capabilities, UniGrasp uses an unsupervised autoencoder to learn a compact latent representation of the gripper's geometry and kinematics. By processing point clouds of the gripper in various configurations (derived from its URDF), the model gains an understanding of the hand's reach, shape, and joint limits. This allows the system to be "hardware-aware" without requiring manual feature engineering for every new gripper.

Point Set Selection Network (PSSN)

The core of the architecture is the Point Set Selection Network (PSSN), which identifies optimal contact points on the object's surface. Given the massive number of possible point combinations, the PSSN employs a multi-stage approach similar to a spatial beam search.

The multi-stage PSSN architecture selects contact points sequentially, ensuring each stage is conditioned on previous selections to maintain grasp stability.
  • Stage 1: Identifies promising initial contact points from the object point cloud.
  • Stage 2: Selects a second point relative to the first, ensuring the pair is viable for the specific gripper.
  • Stage 3: For multi-fingered hands, selects a third point to complete a stable grasp triangle.

Data Generation and Training

The model is trained on a massive synthetic dataset comprising over 3,000 object instances and 12 different robotic grippers. Labels are generated through physics simulations that evaluate two critical criteria: force closure (ensuring the grasp is physically stable) and reachability (verifying that the specific robot hand can physically reach the contact points without collisions).

Results and Evaluation

UniGrasp demonstrates exceptional performance in both simulation and real-world settings. In simulation, the model achieved over 90% accuracy in its top-10 grasp predictions across known grippers.

The most significant result is the model's ability to perform zero-shot generalization to unseen hardware. In real-world experiments, UniGrasp achieved:

  • 93% success with an unseen two-fingered gripper.
  • 90% success with the Allegro hand.
  • 83% success with the complex 5-fingered Schunk SVH.

Compared to traditional heuristic-based planners, which often struggle with high-degree-of-freedom hands (scoring ~40% on the Allegro hand), UniGrasp provides a substantial leap in reliability and versatility.

Conclusion

UniGrasp presents a unified approach to robotic grasping by treating contact points as a common language between objects and grippers. While currently optimized for precision grasps with 2 or 3 fingers, future work aims to extend this to native N-fingered planning and power grasps. By decoupling the grasp strategy from specific hardware, UniGrasp paves the way for more adaptable and general-purpose robotic systems.

Summary updated

Abstract

To achieve a successful grasp, gripper attributes such as its geometry and kinematics play a role as important as the object geometry. The majority of previous work has focused on developing grasp methods that generalize over novel object geometry but are specific to a certain robot hand. We propose UniGrasp, an efficient data-driven grasp synthesis method that considers both the object geometry and gripper attributes as inputs. UniGrasp is based on a novel deep neural network architecture that selects sets of contact points from the input point cloud of the object. The proposed model is trained on a large dataset to produce contact points that are in force closure and reachable by the robot hand. By using contact points as output, we can transfer between a diverse set of multifingered robotic hands. Our model produces over 90% valid contact points in Top10 predictions in simulation and more than 90% successful grasps in real world experiments for various known two-fingered and three-fingered grippers. Our model also achieves 93%, 83% and 90% successful grasps in real world experiments for an unseen two-fingered gripper and two unseen multi-fingered anthropomorphic robotic hands.
BibTeX
@article{shao2020unigrasp,
  author = {Lin Shao and Fábio Ferreira and Mikael Jorda and Varun Nambiar and Jianlan Luo and Eugen Solowjow and Juan Aparicio Ojea and Oussama Khatib and Jeannette Bohg},
  title = {{UniGrasp: Learning a Unified Model to Grasp with N-Fingered Robotic Hands}},
  journal = {IEEE Robotics Autom. Lett.},
  year = {2020},
  doi = {10.1109/lra.2020.2969946},
}
Figure