Abstract
We address the problem of directional semantic grasping, that is, grasping a specific object from a specific direction. We approach the problem using deep reinforcement learning via a double deep Q-network (DDQN) that learns to map downsampled RGB input images from a wrist-mounted camera to Q-values, which are then translated into Cartesian robot control commands via the cross-entropy method (CEM). The network is learned entirely on simulated data generated by a custom robot simulator that models both physical reality (contacts) and perceptual quality (high-quality rendering). The reality gap is bridged using domain randomization. The system is an example of end-to-end (mapping input monocular RGB images to output Cartesian motor commands) grasping of objects from multiple pre-defined object-centric orientations, such as from the side or top. We show promising results in both simulation and the real world, along with some challenges faced and the need for future research in this area.
Introduction
In robotic manipulation, the ability to grasp a specific object from a specific direction—termed Directional Semantic Grasping—is crucial for practical applications. While traditional grasping often focuses on indiscriminate picking, real-world tasks often dictate how an object must be held (e.g., a side-grasp to avoid overhead obstacles). This research, a collaboration between NVIDIA and USC, explores an end-to-end deep reinforcement learning (RL) approach to solve this problem using only monocular RGB images from a wrist-mounted camera, specifically targeting the "sim-to-real" challenge.
Methodology
The core of the system is a Double Deep Q-Network (DDQN) that processes downsampled 64x40 RGB images to predict Q-values. Because the robot operates in a continuous 3D action space, the Cross-Entropy Method (CEM) is utilized to sample and refine the best Cartesian motor commands based on these Q-values.
To overcome the reality gap, the agent is trained entirely within a custom Unreal Engine 4 simulator. This environment leverages NVIDIA PhysX and FleX for realistic contact modeling and high-quality rendering. A key component of the training is domain randomization, where textures, lighting, and camera parameters are varied to prevent the model from overfitting to simulated artifacts and to encourage the learning of robust geometric features.
Experimental Setup
The study utilized four objects from the YCB Dataset with varying properties:
- Meat can (rounded cuboid, reflective)
- Soup can (cylinder, reflective)
- Sugar box (squared cuboid, matte)
- Mustard bottle (unique curved shape, bright color)
The models were trained on 100,000 simulated samples per object. Evaluation was conducted both in simulation and on a physical Baxter robot to test "zero-shot" transfer capabilities.
Results and Discussion
The system achieved successful zero-shot transfer to the real world without any fine-tuning on real data. In simulation, success rates for top-down grasps on the meat can and mustard bottle exceeded 95%.
Real-world performance highlights included:
- Mustard Bottle: 100% success (top), 66.7% (side).
- Meat Can: 83.3% success (top), 75% (side).
- Sugar Box: 50% success (top), 66.7% (side).
- Soup Can: 33.3% success (top), 50% (side).
Several insights emerged during testing. For instance, the sugar box proved difficult for top-down grasps because its featureless white top made distance estimation challenging for an RGB-only network. Additionally, while simulation friction was lower (making the soup can "slippery"), real-world friction actually improved performance for certain grasp types. However, specular reflections on metallic surfaces remained a significant hurdle for the vision system.
Conclusion
This work demonstrates that photorealistic simulation combined with domain randomization and auxiliary learning tasks can enable complex semantic grasping policies to transfer to the real world. Future work will focus on improving sample efficiency, expanding the variety of supported objects, and incorporating temporal data to enhance depth perception and robustness against lighting variations.
Summary updated