Improving Robotic Grasping Ability Through Deep Shape Generation

Junnan Jiang, Yuyang Tu, Xiaohui Xiao, Zhongtao Fu, Jianwei Zhang, Fei Chen, Miao Li

Abstract

Data-driven approaches have become a dominant paradigm for robotic grasp planning. However, the performance of these approaches is enormously influenced by the quality of the available training data. In this paper, we propose a framework to generate object shapes to improve the grasping dataset quality, thus enhancing the grasping ability of a pre-designed learning-based grasp planning network. In this framework, the object shapes are embedded into a low-dimensional feature space using an AutoEncoder (encoder-decoder) based structure network. The rarity and graspness scores are defined for each object shape using outlier detection and grasp-quality criteria. Subsequently, new object shapes are generated in feature space that leverages the original high rarity and graspness score objects' features, which can be employed to augment the grasping dataset. Finally, the results obtained from the simulation and real-world experiments demonstrate that the grasping ability of the learning-based grasp planning network can be effectively improved with the generated object shapes.

Introduction

Data-driven models have revolutionized robotic grasping, yet their efficacy remains tethered to the quality of training data. Existing datasets often suffer from redundancy—over-representing simple, easy-to-grasp shapes like boxes—and significant gaps in "edge cases" where complex geometries lead to robotic failure. Standard augmentation techniques like rotation or scaling fail to introduce fundamentally new geometries. This research introduces a targeted data augmentation framework that identifies rare and difficult-to-grasp objects and interpolates between them in a latent space to generate challenging new training data.

Methodology

The proposed framework follows a three-stage pipeline designed to systematically "fill the gaps" in existing datasets.

Overview of the proposed framework: (1) scoring objects, (2) grouping pairs, and (3) generating new shapes.
  1. Object Shape Encoding (AE-Critic): 3D voxel grids are compressed into a 128-dimensional latent space using an AutoEncoder. To prevent the generation of unrealistic "ghostly" shapes during interpolation, a Critic network acts as a regularizer, ensuring the output remains physically plausible and contiguous.
  2. Data Scoring: Each object is evaluated based on two metrics:
  3. Shape Rarity: Local Outlier Factor detection identifies shapes in underrepresented regions of the feature space.
  4. Graspness Score: Analytical metrics (Dex-Net and Ferrari-Canny) determine the difficulty of finding stable grasps.
  5. Targeted Generation: The system selects pairs of objects that are both rare and difficult to grasp. By interpolating between their feature vectors, the framework generates "hybrid" objects that inherit these challenging properties for dataset augmentation.

Results and Key Insights

The effectiveness of the framework was validated through both simulation and real-world experiments using a Franka Emika Panda robot arm.

Comparison of original and augmented data distributions, showing how generated shapes fill gaps in rarity and graspness.
  • Filling the Latent Void: Visualization via t-SNE demonstrates that the method successfully populates empty regions in the data distribution, specifically targeting areas with high rarity and low graspness.
  • Performance Gains: In real-world tests on eight everyday objects, the robot's success rate improved from 68% (baseline) to 86% after training on the augmented dataset.
  • Quality over Quantity: The researchers noted that blindly increasing data volume can lead to overfitting. Optimal performance requires a careful balance between original data and targeted, high-difficulty generated shapes.
  • Structural Integrity: The inclusion of the Critic network proved essential; it significantly reduced stray voxels compared to standard AutoEncoders, resulting in cleaner training models.

Discussion and Future Work

While the framework significantly boosts grasping success, it currently relies on fixed camera viewpoints and simplified metrics that ignore object scale and orientation. Future work could integrate this into a "lifelong learning" loop, where real-world grasping failures automatically trigger the generation of similar difficult shapes for immediate retraining. This targeted approach ensures that robotic systems continue to evolve alongside the complexity of their environments.

Summary updated

Abstract

Data-driven approaches have become a dominant paradigm for robotic grasp planning. However, the performance of these approaches is enormously influenced by the quality of the available training data. In this paper, we propose a framework to generate object shapes to improve the grasping dataset quality, thus enhancing the grasping ability of a pre-designed learning-based grasp planning network. In this framework, the object shapes are embedded into a low-dimensional feature space using an AutoEncoder (encoder-decoder) based structure network. The rarity and graspness scores are defined for each object shape using outlier detection and grasp-quality criteria. Subsequently, new object shapes are generated in feature space that leverages the original high rarity and graspness score objects' features, which can be employed to augment the grasping dataset. Finally, the results obtained from the simulation and real-world experiments demonstrate that the grasping ability of the learning-based grasp planning network can be effectively improved with the generated object shapes.
BibTeX
@misc{jiang2022improving,
  author = {Junnan Jiang and Yuyang Tu and Xiaohui Xiao and Zhongtao Fu and Jianwei Zhang and Fei Chen and Miao Li},
  title = {{Improving Robotic Grasping Ability Through Deep Shape Generation}},
  year = {2022},
  eprint = {2206.09353},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2206.09353},
}
Figure