Grasp success prediction with quality metrics

Carlos Rubert, Daniel Kappler, Jeannette Bohg, Antonio MoralesarXiv 2018 · arXiv

Introduction: Bridging the Sim-to-Real Gap

In robotic manipulation, a "grasp" is far more complex than simply closing a gripper. It involves intricate interactions between geometry and physics. For decades, researchers have relied on mathematical "quality metrics"—such as the area of a grasp polygon or the volume of a convex hull—to estimate grasp stability in simulation. However, a significant "sim-to-real" gap persists: grasps that appear optimal in a simulator often fail in physical environments due to sensor noise, misalignments, or unmodeled object dynamics.

This research investigates which geometric metrics actually correlate with real-world success and demonstrates how machine learning can synthesize these metrics into a reliable predictor of robotic performance.

Methodology: Beyond Binary Classification

The researchers moved beyond the traditional "pass/fail" binary by introducing a more nuanced three-tier classification system:

  • Robust: The grasp consistently succeeds despite minor variations.
  • Futile: The grasp consistently fails.
  • Fragile: The grasp is highly sensitive, succeeding or failing based on minute changes in contact points or orientation.
Examples of robust (green check) versus futile or fragile (red cross) grasps on a block and a hammer.

To predict these categories, the study narrowed down ten common metrics to seven independent features, including the smallest singular value of the grasp matrix, distance to the center of mass, and joint posture. These features served as inputs for two machine learning classifiers: k-Nearest Neighbors (k-NN) and Classification Trees.

Experimental Setup and Datasets

The study's validation involved a rigorous 10-step physical protocol across two distinct robotic platforms: the Apollo robot (equipped with a Schunk SDH hand) and the Tombatossals robot (equipped with a Barrett hand).

The two robotic hands used in the study: (a) Barrett hand and (b) Schunk SDH hand.

Testing was conducted on nine 3D-printed models of household items, such as bottles, toasters, and cameras. These physical trials were compared against a library of approximately 900,000 simulated grasp configurations generated via the OpenHand simulator.

Key Insights and Results

The research yielded several critical findings:

  1. Synergy of Metrics: No single geometric metric is a "silver bullet." For instance, the Inverse Condition Number ($Q_{D2}$) performed only slightly better than random chance. However, when combined via Classification Trees, the predictive accuracy reached 76%.
  2. The Importance of Posture: The "Posture of manipulator joints" ($Q_{D1}$) emerged as a surprisingly strong individual predictor, indicating that the internal configuration of the robot hand is as vital as the contact points on the object.
  3. Classification Accuracy: The 3-category scale consistently outperformed binary models. In the combined dataset, the 3-category model reached 76% accuracy compared to 71% for the binary k-NN model.

Conclusion and Future Directions

While combining geometric metrics significantly improves grasp prediction, limitations remain. Current models do not account for material properties like friction or elasticity. Future work will explore whether these findings generalize to simpler parallel-jaw grippers and how they perform in cluttered environments where reachability constraints are more severe.

Summary updated

Abstract

Current robotic manipulation requires reliable methods to predict whether a certain grasp on an object will be successful or not prior to its execution. Different methods and metrics have been developed for this purpose but there is still work to do to provide a robust solution. In this article we combine different metrics to evaluate real grasp executions. We use different machine learning algorithms to train a classifier able to predict the success of candidate grasps. Our experiments are performed with two different robotic grippers and different objects. Grasp candidates are evaluated in both simulation and real world. We consider 3 different categories to label grasp executions: robust, fragile and futile. Our results shows the proposed prediction model has success rate of 76\%.
BibTeX
@misc{rubert2018grasp,
  author = {Carlos Rubert and Daniel Kappler and Jeannette Bohg and Antonio Morales},
  title = {{Grasp success prediction with quality metrics}},
  year = {2018},
  eprint = {1809.03276},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1809.03276},
}
Figure