Hybrid Physical Metric For 6-DoF Grasp Pose Detection

Yuhao Lu, Beixing Deng, Zhenyu Wang, Peiyuan Zhi, Yali Li, Shengjin Wang

TL;DR

This paper introduces a novel "hybrid physical metric" that combines multiple physical properties (flatness, gravity, collision) with the traditional force-closure metric to provide more nuanced grasp confidence scores. They also propose FGC-GraspNet, a multi-resolution neural network designed to effectively learn these refined scores, leading to significantly improved 6-DoF grasp pose detection in real-world cluttered environments.

What Problem Does This Paper Address?

Grasping objects is a fundamental task for robots, but accurately predicting how to grasp an object (6-DoF grasp pose detection) in complex, cluttered environments remains a significant challenge. Current data-driven methods rely on neural networks to predict grasp poses, but their performance is heavily influenced by how they evaluate the "quality" or "confidence" of a potential grasp. Existing methods often use a single physical metric, like force-closure, which provides only a few discrete confidence levels. This coarse evaluation can lead to millions of grasp poses being assigned the same confidence score, making it difficult for the network to distinguish between good and bad grasps and resulting in inaccurate predictions. The paper aims to address this "evaluation insufficiency" by developing a more sophisticated way to measure grasp quality.

Methods

The core innovation is the hybrid physical metric. Instead of relying solely on the force-closure metric, the authors propose a new metric that integrates:

  • Flatness Metric: Measures how flat the contact surface between the gripper and the object is. This is quantified by analyzing the similarity of local normal vectors and the perpendicularity between the antipodal direction and the contact region.
  • Gravity Center Metric: Assesses how well the grasp's antipodal force aligns with the object's gravity center. A closer alignment, measured by the Euclidean distance to the antipodal line, indicates a more stable grasp.
  • Collision Perturbation Metric: Aims to prevent grasps that are too close to the object's surface, which could lead to collisions. It's calculated as the minimum Euclidean distance between gripper end points and object contact points.

These three new metrics are combined with the traditional force-closure metric (which provides a 10-bin score) using a weighted sum to produce a more refined, continuous grasp confidence score.

To learn these new scores effectively, the authors designed FGC-GraspNet, a multi-resolution neural network. It uses a PointNet++ backbone to extract features at different resolutions. A "Foreground-Approach-head" (FA-head) handles lower-resolution features for tasks like foreground segmentation and approach direction prediction. A "Rotation-Depth-head" (RD-head) utilizes higher-resolution features and a novel local attention module to capture intricate local geometric information crucial for in-plane rotation and depth prediction. This module uses cylindrical region queries and self-attention to integrate spatial neighbor features. The network also employs a new joint loss function that regresses all predicted scores (rather than just the maximum) and includes a specific loss for depth prediction.

Datasets

  • GraspNet-1Billion dataset [3]: Used for training and evaluating the proposed hybrid physical metric and FGC-GraspNet. This dataset provides 6-DoF grasp poses annotated on object models.
  • Real-world datasets (Realsense/Kinect): Used for evaluating the method's performance on real robotic experiments.

Key Insights and Ideas

  • The inadequacy of single physical metrics: The paper convincingly demonstrates that relying on a single metric like force-closure is insufficient for distinguishing between a vast number of potential grasp poses, leading to noisy confidence scores and poor network performance.
  • Human-like reasoning for grasp quality: The hybrid metric's components (flatness, gravity alignment, collision avoidance) are inspired by how humans intuitively assess grasp stability.
  • Multi-resolution learning is key: Different aspects of grasp prediction (e.g., overall scene understanding vs. fine local geometry) benefit from features at different resolutions. FGC-GraspNet's architecture effectively leverages this.
  • Local attention for fine-grained geometry: The proposed local attention module is crucial for capturing the detailed geometric cues needed for accurate rotation and depth prediction, which are often overlooked by methods that only consider global features.
  • Regressing all scores is better: Instead of just predicting the single "best" grasp score, regressing all candidate scores provides richer supervision for the network.

Key Results

  • Improved grasp confidence: The hybrid physical metric generates more nuanced and physically meaningful confidence scores, better reflecting real-world grasp success.
  • State-of-the-art performance: FGC-GraspNet, trained with the hybrid metric, achieves significantly higher Average Precision (AP) scores compared to previous methods on the GraspNet-1Billion dataset, outperforming the baseline [3] by over 10% on seen test sets.
  • Real-world success: The method demonstrates strong performance in real robotic experiments, achieving a 90.5% success rate in cluttered scenes. Qualitative examples show grasps predicted with the hybrid metric are more stable and exhibit characteristics like flat contact points and better gravity alignment.
  • Ablation study highlights depth loss: The inclusion of a depth classification loss in the joint loss function significantly improves performance, demonstrating its importance.

Limitations and Future Work

  • Limitations: The paper acknowledges that the force-closure metric still contributes to the hybrid metric, suggesting potential for further refinement by exploring even more comprehensive physical properties. The discrete nature of some input data (e.g., object models) might also limit ultimate performance.
  • Future Work: The authors express interest in extending their framework to more complex integrated robotic tasks such as feeding, cooking, or table cleaning, indicating a desire to move beyond simple object grasping.

Summary updated

Abstract

6-DoF grasp pose detection of multi-grasp and multi-object is a challenge task in the field of intelligent robot. To imitate human reasoning ability for grasping objects, data driven methods are widely studied. With the introduction of large-scale datasets, we discover that a single physical metric usually generates several discrete levels of grasp confidence scores, which cannot finely distinguish millions of grasp poses and leads to inaccurate prediction results. In this paper, we propose a hybrid physical metric to solve this evaluation insufficiency. First, we define a novel metric is based on the force-closure metric, supplemented by the measurement of the object flatness, gravity and collision. Second, we leverage this hybrid physical metric to generate elaborate confidence scores. Third, to learn the new confidence scores effectively, we design a multi-resolution network called Flatness Gravity Collision GraspNet (FGC-GraspNet). FGC-GraspNet proposes a multi-resolution features learning architecture for multiple tasks and introduces a new joint loss function that enhances the average precision of the grasp detection. The network evaluation and adequate real robot experiments demonstrate the effectiveness of our hybrid physical metric and FGC-GraspNet. Our method achieves 90.5\% success rate in real-world cluttered scenes. Our code is available at https://github.com/luyh20/FGC-GraspNet.
BibTeX
@inproceedings{lu2022hybrid,
  author = {Yuhao Lu and Beixing Deng and Zhenyu Wang and Peiyuan Zhi and Yali Li and Shengjin Wang},
  title = {{Hybrid Physical Metric For 6-DoF Grasp Pose Detection}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2022},
  url = {https://openreview.net/forum?id=tnzT1uElOk},
}
Figure