Toward Sim-to-Real Directional Semantic Grasping

Shariq Iqbal, Jonathan Tremblay, Andy Campbell, Kirby Leung, Thang To, Jia Cheng, Erik Leitch, Duncan McKay, Stan Birchfield

Abstract

We address the problem of directional semantic grasping, that is, grasping a specific object from a specific direction. We approach the problem using deep reinforcement learning via a double deep Q-network (DDQN) that learns to map downsampled RGB input images from a wrist-mounted camera to Q-values, which are then translated into Cartesian robot control commands via the cross-entropy method (CEM). The network is learned entirely on simulated data generated by a custom robot simulator that models both physical reality (contacts) and perceptual quality (high-quality rendering). The reality gap is bridged using domain randomization. The system is an example of end-to-end (mapping input monocular RGB images to output Cartesian motor commands) grasping of objects from multiple pre-defined object-centric orientations, such as from the side or top. We show promising results in both simulation and the real world, along with some challenges faced and the need for future research in this area.

Introduction

In robotic manipulation, the ability to grasp a specific object from a specific direction—termed Directional Semantic Grasping—is crucial for practical applications. While traditional grasping often focuses on indiscriminate picking, real-world tasks often dictate how an object must be held (e.g., a side-grasp to avoid overhead obstacles). This research, a collaboration between NVIDIA and USC, explores an end-to-end deep reinforcement learning (RL) approach to solve this problem using only monocular RGB images from a wrist-mounted camera, specifically targeting the "sim-to-real" challenge.

Methodology

The core of the system is a Double Deep Q-Network (DDQN) that processes downsampled 64x40 RGB images to predict Q-values. Because the robot operates in a continuous 3D action space, the Cross-Entropy Method (CEM) is utilized to sample and refine the best Cartesian motor commands based on these Q-values.

Simulated wrist-mounted camera views illustrating the domain randomization used to bridge the reality gap.

To overcome the reality gap, the agent is trained entirely within a custom Unreal Engine 4 simulator. This environment leverages NVIDIA PhysX and FleX for realistic contact modeling and high-quality rendering. A key component of the training is domain randomization, where textures, lighting, and camera parameters are varied to prevent the model from overfitting to simulated artifacts and to encourage the learning of robust geometric features.

Experimental Setup

The study utilized four objects from the YCB Dataset with varying properties:

  1. Meat can (rounded cuboid, reflective)
  2. Soup can (cylinder, reflective)
  3. Sugar box (squared cuboid, matte)
  4. Mustard bottle (unique curved shape, bright color)

The models were trained on 100,000 simulated samples per object. Evaluation was conducted both in simulation and on a physical Baxter robot to test "zero-shot" transfer capabilities.

Results and Discussion

The system achieved successful zero-shot transfer to the real world without any fine-tuning on real data. In simulation, success rates for top-down grasps on the meat can and mustard bottle exceeded 95%.

Real-world experimental setup featuring the Baxter robot and the corresponding view from its wrist-mounted camera.

Real-world performance highlights included:

  • Mustard Bottle: 100% success (top), 66.7% (side).
  • Meat Can: 83.3% success (top), 75% (side).
  • Sugar Box: 50% success (top), 66.7% (side).
  • Soup Can: 33.3% success (top), 50% (side).

Several insights emerged during testing. For instance, the sugar box proved difficult for top-down grasps because its featureless white top made distance estimation challenging for an RGB-only network. Additionally, while simulation friction was lower (making the soup can "slippery"), real-world friction actually improved performance for certain grasp types. However, specular reflections on metallic surfaces remained a significant hurdle for the vision system.

Conclusion

This work demonstrates that photorealistic simulation combined with domain randomization and auxiliary learning tasks can enable complex semantic grasping policies to transfer to the real world. Future work will focus on improving sample efficiency, expanding the variety of supported objects, and incorporating temporal data to enhance depth perception and robustness against lighting variations.

Summary updated

Abstract

We address the problem of directional semantic grasping, that is, grasping a specific object from a specific direction. We approach the problem using deep reinforcement learning via a double deep Q-network (DDQN) that learns to map downsampled RGB input images from a wrist-mounted camera to Q-values, which are then translated into Cartesian robot control commands via the cross-entropy method (CEM). The network is learned entirely on simulated data generated by a custom robot simulator that models both physical reality (contacts) and perceptual quality (high-quality rendering). The reality gap is bridged using domain randomization. The system is an example of end-to-end (mapping input monocular RGB images to output Cartesian motor commands) grasping of objects from multiple pre-defined object-centric orientations, such as from the side or top. We show promising results in both simulation and the real world, along with some challenges faced and the need for future research in this area.
BibTeX
@misc{iqbal2019toward,
  author = {Shariq Iqbal and Jonathan Tremblay and Andy Campbell and Kirby Leung and Thang To and Jia Cheng and Erik Leitch and Duncan McKay and Stan Birchfield},
  title = {{Toward Sim-to-Real Directional Semantic Grasping}},
  year = {2019},
  eprint = {1909.02075},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1909.02075},
}
Figure