Generating Grasp Poses for a High-DOF Gripper Using Neural Networks

Min Liu, Zherong Pan, Kai Xu, Kanishka Ganguly, Dinesh Manocha

Introduction

Robotic grasping research has traditionally focused on simple parallel-jaw grippers with limited degrees of freedom (DOF). While effective for basic tasks, these grippers lack the dexterity required for complex, human-like manipulation. Transitioning to high-DOF robotic hands, such as the 25-DOF Shadow Hand, offers significant versatility but introduces a major computational hurdle: pose ambiguity.

Pose ambiguity occurs because there are hundreds of equally valid ways for a complex hand to grasp a single object. Standard supervised learning models often struggle with this redundancy, as they try to force a neural network to map an object to one specific "ground truth" grasp. This often leads to inconsistent results or a failure to converge during training. This paper presents a deep learning approach designed to overcome these challenges using a novel consistency loss function and collision-aware training.

Methodology

The researchers proposed a direct mapping approach where a neural network takes a 3D observation of an object and immediately outputs a grasp pose. The system architecture is divided into two primary components:

  1. Object Feature Extraction ($NN_o$): A 3D convolutional network that converts a 40x40x40 occupancy grid of the object into a low-dimensional feature vector.
  2. Pose Generation ($NN_x$): A fully connected network that maps that vector to the specific joint angles and wrist transformations of the robotic hand.

Key Innovations

To make this architecture effective for high-DOF hands, the authors introduced three critical innovations:

  • Consistency Loss: Instead of training the network on a single "best" grasp, the model is provided with a set of $K$ candidate grasps for each object. During training, the network is allowed to minimize the distance to the candidate closest to its current prediction. This allows the network to "choose" the most representable grasp, resolving the ambiguity of multiple correct answers.
  • Collision Loss: Utilizing a Signed Distance Field (SDF), the network is penalized if the predicted hand pose intersects with the object's volume. This effectively "pushes" the fingers toward the surface of the object rather than through it.
  • Runtime Refinement: After the network makes an initial prediction (taking approximately 0.54s), a quick optimization step (0.46s) nudges the pose to ensure zero collisions and optimal contact.

Dataset and Training

The model was trained on a massive dataset of 324 mesh models compiled from the BigBIRD Database, KIT Object Models, YCB Benchmarks, and the Grasp Database. To ensure the network was invariant to object orientation, the researchers employed extensive data augmentation, rotating each object and its 100 best grasps along 27 different axes. This resulted in a dataset 27 times larger than the original collection.

Results

The proposed method demonstrated significant improvements in both accuracy and practical utility:

  • Accuracy: The approach achieved 4x higher accuracy in terms of distance to ground truth poses compared to standard supervised learning baselines.
  • Collision Reduction: The collision loss reduced average penetration depth from 0.0553m to just 0.0081m prior to the final refinement step.
  • Speed: A complete grasp can be generated in roughly 3-5 seconds at runtime, making real-time dexterous grasping feasible.
  • Real-World Success: In physical tests using a Shadow Hand mounted on a UR10 arm, the system successfully grasped 12 out of 15 previously unseen objects. The system proved robust even when using noisy 3D models reconstructed from multi-view depth cameras.

Limitations and Future Work

Despite its success, the system has limitations regarding "reachability." Currently, the model predicts a hand pose but does not guarantee that the robotic arm can reach that pose without environmental collisions. Additionally, the researchers noted that "high-genus" objects (objects with holes, like donuts) present geometric challenges that a simple SDF might not fully capture.

Future research will focus on moving toward a truly end-to-end architecture that generates grasp poses directly from raw camera images, bypassing the intermediate 3D mesh reconstruction phase.

Summary updated

Abstract

We present a learning-based method for representing grasp poses of a high-DOF hand using neural networks. Due to redundancy in such high-DOF grippers, there exists a large number of equally effective grasp poses for a given target object, making it difficult for the neural network to find consistent grasp poses. We resolve this ambiguity by generating an augmented dataset that covers many possible grasps for each target object and train our neural networks using a consistency loss function to identify a one-to-one mapping from objects to grasp poses. We further enhance the quality of neural-network-predicted grasp poses using a collision loss function to avoid penetrations. We use an object dataset that combines the BigBIRD Database, the KIT Database, the YCB Database, and the Grasp Dataset to show that our method can generate high-DOF grasp poses with higher accuracy than supervised learning baselines. The quality of the grasp poses is on par with the groundtruth poses in the dataset. In addition, our method is robust and can handle noisy object models such as those constructed from multi-view depth images, allowing our method to be implemented on a 25-DOF Shadow Hand hardware platform.
BibTeX
@inproceedings{liu2019generating,
  author = {Min Liu and Zherong Pan and Kai Xu and Kanishka Ganguly and Dinesh Manocha},
  title = {{Generating Grasp Poses for a High-DOF Gripper Using Neural Networks}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2019},
  doi = {10.1109/iros40897.2019.8968115},
}
Figure