Directional Semantic Grasping of Real-World Objects: From Simulation to Reality

Shariq Iqbal, Jonathan Tremblay, Thang To, Jia Cheng, Erik Leitch, Andy Campbell, Kirby Leung, Duncan McKay, Stan Birchfield

Abstract

We address the problem of directional semantic grasping, that is, grasping a specific object from a specific direction. We approach the problem using deep reinforcement learning via a double deep Q-network (DDQN) that learns to map downsampled RGB input images from a wrist-mounted camera to Q-values, which are then translated into Cartesian robot control commands via the cross-entropy method (CEM). The network is learned entirely on simulated data generated by a custom robot simulator that models both physical reality (contacts) and perceptual quality (high-quality rendering). The reality gap is bridged using domain randomization. The system is an example of end-to-end grasping of objects from multiple pre-defined object-centric orientations, such as from the side or top. We show promising results in both simulation and the real world, along with some challenges faced and the need for future research in this area.

Introduction

Most robotic grasping research focuses on "indiscriminate" grasping—simply picking up any object in a cluttered environment, typically using a top-down approach. However, real-world applications often require semantic knowledge (identifying a specific object) and directional precision (approaching from a specific angle). For instance, placing a bottle on a shelf with limited overhead clearance requires a side-grasp rather than a top-down one. This paper introduces a method for "directional semantic grasping," enabling a robot to pick up a specific object from a pre-defined orientation while overcoming the "reality gap" between simulation and the physical world.

Methodology

The proposed system utilizes an end-to-end Deep Reinforcement Learning (RL) pipeline based on a Double Deep Q-Network (DDQN). The input consists of downsampled 64x40 RGB images from a wrist-mounted camera. To navigate a continuous action space, the Cross-Entropy Method (CEM) is employed to sample and identify the optimal 4-DOF action, encompassing 3D Cartesian translation and 1D rotation about the wrist axis.

Comparison of simulation and real-world environments from external and gripper camera perspectives.

To accelerate learning, auxiliary prediction heads were integrated into the network to estimate the distance to the object's centroid and its rotational offset. These auxiliary tasks provide a denser learning signal than sparse grasp rewards alone. Training was conducted in a custom simulator built on Unreal Engine 4 (UE4), utilizing NVIDIA PhysX and FleX for physics and contact modeling. To facilitate zero-shot transfer to the real world, heavy domain randomization was applied, including variations in textures, lighting, and the application of Gaussian blur to simulated images.

Experimental Setup

The study utilized objects from the YCB dataset, selected for their diverse geometries and textures: a meat can (rounded cuboid, reflective), a soup can (cylinder, reflective), a sugar box (squared cuboid, matte), and a mustard bottle (curved, bright color). The models were trained on 100,000 simulated samples per object. Evaluation was performed in both a held-out simulated environment and a real-world setup using a Baxter robot.

Results and Discussion

The system demonstrated successful zero-shot transfer from simulation to reality without any fine-tuning on real-world data. The mustard bottle achieved a 100% success rate for top-down grasps and 66.7% for side grasps in real-world trials. The meat can showed 83.3% success for top-down and 75% for side grasps. Interestingly, the soup can performed better in the real world (50% side grasp) than in simulation, likely due to more favorable real-world friction characteristics.

Several key insights emerged during development:

  • Auxiliary Losses: Predicting object distance and orientation as side-tasks was essential for model convergence.
  • Physics Exploits: Early agents learned to exploit the physics engine by applying excessive force to "snap" objects into the gripper. This was mitigated by introducing a reward penalty for high contact forces.
  • Visual Features: The sugar box was easier to grasp from the side than the top because its textured sides provided more visual information than its featureless white top.
  • Ablation Studies: Removing image augmentation or auxiliary targets significantly degraded performance, confirming their necessity for robust sim-to-real transfer.

Conclusion

While the results are promising, challenges remain. Reflective surfaces, such as metallic can tops, are difficult to model accurately in simulation. Furthermore, the current approach is limited to a small set of known objects and relies on single-frame inputs. Future work will focus on scaling to unseen objects, improving friction and specular reflection modeling, and incorporating temporal data to allow for real-time adjustments during the grasping process.

Summary updated

Abstract

We address the problem of directional semantic grasping, that is, grasping a specific object from a specific direction. We approach the problem using deep reinforcement learning via a double deep Q-network (DDQN) that learns to map downsampled RGB input images from a wrist-mounted camera to Q-values, which are then translated into Cartesian robot control commands via the cross-entropy method (CEM). The network is learned entirely on simulated data generated by a custom robot simulator that models both physical reality (contacts) and perceptual quality (high-quality rendering). The reality gap is bridged using domain randomization. The system is an example of end-to-end (mapping input monocular RGB images to output Cartesian motor commands) grasping of objects from multiple pre-defined object-centric orientations, such as from the side or top. We show promising results in both simulation and the real world, along with some challenges faced and the need for future research in this area.
BibTeX
@misc{iqbal2019directional,
  author = {Shariq Iqbal and Jonathan Tremblay and Thang To and Jia Cheng and Erik Leitch and Andy Campbell and Kirby Leung and Duncan McKay and Stan Birchfield},
  title = {{Directional Semantic Grasping of Real-World Objects: From Simulation to Reality}},
  year = {2019},
  eprint = {1909.02075},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1909.02075},
}
Figure