Deformation-Aware Data-Driven Grasp Synthesis

Tran Nguyen Le, Jens Lundell, Fares Abu-Dakka, Ville KyrkiIEEE RA-L 2022 · OpenReview

TL;DR

This paper introduces a novel deep learning approach for robotic grasping that leverages object stiffness information to synthesize more effective grasps for deformable objects. By incorporating stiffness into a standard grasp synthesis pipeline, the authors demonstrate significant improvements in grasp success rates in both simulation and real-world experiments, enabling robots to adapt their grasping strategies to object properties.

What Problem Does This Paper Address?

Traditional robotic grasping methods often assume objects are rigid and don't deform. This assumption breaks down when dealing with everyday objects like sponges, soft toys, or even food, where deformation is common and can even be exploited by humans for better manipulation. While some research has focused on minimizing deformation during grasping, this paper tackles the challenge of harnessing object deformation by considering an object's stiffness. The core problem is how to generate high-quality grasps for objects with varying degrees of deformability, moving beyond the rigid object paradigm.

Methods

The authors propose Deformation-Aware Generative Grasping Convolutional Neural Network (Def-GG-CNN). This is an extension of a state-of-the-art deep grasp planning network (GG-CNN). The key innovation is the addition of an object stiffness image as an input alongside the standard depth image. This stiffness image, representing the object's Young's modulus at each pixel, allows the network to learn how stiffness influences grasp quality and configuration. The network outputs a grasp map containing grasp quality, orientation, and gripper width for each pixel. The training process utilizes a synthetic dataset generated in the Isaac Gym simulator, which allows for precise control over object material properties.

Datasets

  • Newly Curated Synthetic Dataset: The authors created a novel dataset for training their Def-GG-CNN. This dataset consists of depth images, corresponding stiffness images, and ground-truth grasp maps. The data was generated using the Isaac Gym simulator, featuring 120 unique object configurations (30 base objects with 4 stiffness values each). The objects included primitives, YCB dataset objects, and adversarial geometry objects.
  • Evaluation Datasets:
    • Simulation: 7 common objects (shown in Fig. 5) and 28 adversarial objects from the EGAD! dataset.
  • Real-World: 14 objects with varying sizes, shapes, and stiffnesses (shown in Fig. 7).

Key Insights and Ideas

  • Stiffness is Crucial for Deformable Grasping: The paper strongly suggests that ignoring object stiffness leads to suboptimal grasp performance, especially for deformable objects.
  • Exploiting Deformation for Better Grasps: Instead of solely trying to minimize deformation, the approach learns to leverage it. This allows for more robust grasps, particularly for complex shapes.
  • Adaptive Grasping Strategies: The network can learn to generate different grasp types (e.g., pinching for soft objects, caging for rigid ones) based on stiffness, even without explicit pinch grasps in the training data.
  • Synthetic Data Transferability: The research demonstrates that a model trained purely on synthetic data can generalize well to real-world grasping tasks, a significant step for data-driven robotics.

Key Results

  • Simulation: The Def-GG-CNN achieved a significantly higher grasp success rate compared to a baseline that ignored stiffness. For the shake task, improvements were 35% on common objects and 11% on EGAD! objects. The approach also showed better robustness to changes in stiffness values.
  • Real-World: The proposed method achieved approximately 17% higher grasp success rate compared to scenarios where stiffness was assumed to be either uniformly high or uniformly low.
  • Qualitative Findings: The paper shows examples of how the robot adapts its grasping strategy: pinching soft sponges and caging harder objects. It also highlights cases where assuming rigidity for deformable objects leads to failure, and vice-versa.

Limitations and Future Work

  • Computational Cost: The use of FEM simulations for data generation is computationally expensive, which might limit the scalability for real-time applications.
  • Generalizability to Non-Uniform Stiffness: While the approach can handle varying stiffness, its ability to generalize to objects with highly non-uniform internal stiffness distributions is still an open question.
  • Future Work: The authors suggest exploring analytical quality measures that exploit deformations and investigating the use of more efficient simulation methods. They also highlight the potential for developing systems that can account for all variability captured in training data, with simulation quality and efficiency being central bottlenecks.

Summary updated

Abstract

Grasp synthesis for 3D deformable objects remains a little-explored topic, most works aiming to minimize deformations. However, deformations are not necessarily harmful---humans are, for example, able to exploit deformations to generate new potential grasps. How to achieve that on a robot is though an open question. This paper proposes an approach that uses object stiffness information in addition to depth images for synthesizing high-quality grasps. We achieve this by incorporating object stiffness as an additional input to a state-of-the-art deep grasp planning network. We also curate a new synthetic dataset of grasps on objects of varying stiffness using the Isaac Gym simulator for training the network. We experimentally validate and compare our proposed approach against the case where we do not incorporate object stiffness on a total of 2800 grasps in simulation and 560 grasps on a real Franka Panda Emika. The experimental results show significant improvement in grasp success rate using the proposed approach on a wide range of objects with varying shapes, sizes, and stiffnesses. Furthermore, we demonstrate that the approach can generate different grasping strategies for different stiffness values. Together, the results clearly show the value of incorporating stiffness information when grasping objects of varying stiffness. Code and video are available at: https://irobotics.aalto.fi/defggcnn.
BibTeX
@article{le2022deformation,
  author = {Tran Nguyen Le and Jens Lundell and Fares Abu-Dakka and Ville Kyrki},
  title = {{Deformation-Aware Data-Driven Grasp Synthesis}},
  journal = {IEEE Robotics and Automation Letters},
  year = {2022},
  url = {https://openreview.net/forum?id=LsbvNABod2},
}
Figure