Scoring Graspability based on Grasp Regression for Better Grasp Prediction

Amaury Depierre, Emmanuel Dellandréa, Liming Chen

TL;DR

This paper introduces a novel deep learning architecture for robotic grasp prediction that improves accuracy by explicitly correlating grasp parameter regression with a graspability score. By adding a "scorer" module and a new loss function, the method achieves state-of-the-art results on benchmark datasets and demonstrates significant improvements in real-world robotic grasping.

What Problem Does This Paper Address?

Robots need to be adept at grasping objects to interact effectively with their environment. Current deep learning approaches often predict grasp parameters (like position, size, and orientation) and a graspability score independently. This separation can lead to suboptimal grasp predictions, where a seemingly good grasp location might fail due to subtle errors in the predicted parameters, especially when dealing with complex object geometries or occlusions. The challenge is to create a system that not only finds potential grasp locations but also ensures these locations are truly viable given the predicted grasp parameters, bridging the gap between simulated performance and real-world success.

Methods

The authors propose a three-component deep neural network architecture:

  1. Feature Extractor (FE): A ResNet-50 network is used to extract visual features from input RGB images.
  2. Primary Grasp Predictor (PGP): This component, based on a state-of-the-art oriented anchor box approach, predicts grasp parameters (as deformations of reference anchors) and a primary grasp quality score. Crucially, this primary score is predicted independently of the final grasp parameters.
  3. Scorer: This is the novel addition. It takes the visual features and the predicted grasp parameters from PGP as input to refine the grasp quality score. This explicit correlation between grasp parameters and their quality is the core innovation.

A new loss function is introduced that links the regression loss for grasp parameters with the output of the scorer. This encourages the network to learn grasp parameters that are not only accurate but also lead to high scores from the scorer.

Datasets

  • Cornell Grasping Dataset: A widely used dataset for grasp detection.
  • Jacquard Dataset: A larger dataset (54k images) used for more extensive evaluation. The authors note that their cross-validation was performed object-wise to ensure images with the same object were either all in training or all in testing.

Key Insights and Ideas

  • The Importance of Correlation: The paper strongly argues that predicting grasp parameters and their quality independently is a bottleneck. Explicitly correlating them, as done by the scorer, leads to more robust and accurate grasp predictions.
  • Scorer as a Refinement Mechanism: The scorer acts like a second opinion, validating the predicted grasp parameters against visual context, leading to a more reliable final grasp quality score.
  • Anchor Box Optimization: Using a mean grasp box derived from the dataset, rather than arbitrary ratios, improved performance and reduced hyperparameter tuning.
  • Grasp Selection for Training: Using the primary grasp score from PGP to select the most promising grasps for training the scorer is more effective than processing all proposals, as it focuses training on potentially challenging but relevant examples.

Key Results

  • Jacquard Dataset: The proposed method achieved an accuracy of 85.74%, outperforming the previous state-of-the-art by 1.74 points. This is a significant improvement over a baseline without the scorer (82.13%).
  • Cornell Dataset: The model achieved 95.2% accuracy, comparable to existing state-of-the-art methods.
  • Real Robot Testing: On a Fanuc robotic arm, the full model achieved 92.4% accuracy, a substantial improvement over the baseline without the scorer (88.1%). This demonstrates the method's effectiveness in bridging the sim-to-real gap.

Limitations and Future Work

  • Limitations: The paper acknowledges that the primary grasp score from PGP is less accurate than the refined score from the scorer, highlighting the scorer's necessity. The performance on the Cornell dataset is only marginally behind the best existing methods, suggesting room for further improvement.
  • Future Work: The authors plan to explore meta-learning techniques to enable grasp prediction models to adapt quickly to new situations, implying a desire for more generalized and adaptable grasping systems.

Summary updated

Abstract

Grasping objects is one of the most important abilities that a robot needs to master in order to interact with its environment. Current state-of-the-art methods rely on deep neural networks trained to jointly predict a graspability score together with a regression of an offset with respect to grasp reference parameters. However, these two predictions are performed independently, which can lead to a decrease in the actual graspability score when applying the predicted offset. Therefore, in this paper, we extend a state-of-the-art neural network with a scorer that evaluates the graspability of a given position, and introduce a novel loss function which correlates regression of grasp parameters with graspability score. We show that this novel architecture improves performance from 82.13% for a state-of-the-art grasp detection network to 85.74% on Jacquard dataset. When the learned model is transferred onto a real robot, the proposed method correlating graspability and grasp regression achieves a 92.4% rate compared to 88.1% for the baseline trained without the correlation.
BibTeX
@inproceedings{depierre2021scoring,
  author = {Amaury Depierre and Emmanuel Dellandréa and Liming Chen},
  title = {{Scoring Graspability based on Grasp Regression for Better Grasp Prediction}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2021},
  doi = {10.1109/icra48506.2021.9561198},
}

Loading related content...

Figure