Learning 6-DoF Grasping and Pick-Place Using Attention Focus

Marcus Gualtieri, Robert Platt Jr.

Abstract

This paper introduces a method for robots to learn complex 6-Degree-of-Freedom (6-DoF) pick-and-place tasks by focusing their "attention" through a hierarchical sequence of gazes. By narrowing the massive search space of 3D positions and orientations into a series of zoomed-in sensing actions, the system enables deep reinforcement learning to master manipulation of novel objects in clutter, with successful zero-shot transfer from simulation to real-world hardware.

Introduction

Robotic manipulation in the real world is notoriously difficult for Reinforcement Learning (RL). While RL excels in games with discrete actions, physical robots operate in a high-dimensional, continuous 6-DoF action space (3D position + 3D orientation). Furthermore, the "state" of the world—cluttered scenes with novel objects and partial sensor data—is incredibly complex.

Most existing solutions take shortcuts, such as limiting the robot to top-down 2D grasping or assuming the robot already knows the 3D models of the objects. This paper tackles the "full" problem: how can a robot learn to grasp any object and place it in a specific non-trivial orientation (like putting a bottle upright on a coaster) using only raw depth sensor data?

Methodology

Sense-Move-Effect MDP

The authors reformulate the problem as a Markov Decision Process where the robot alternates between "sensing" (looking at the scene) and "moving/effecting" (executing the grasp or place). Instead of raw joint torques, the actions are abstract target poses. This abstraction allows the agent to focus on the geometry of the task rather than the low-level control of the arm.

The Sense-Move-Effect MDP framework and workspace setup.

Hierarchical SE(3) Sampling (HSE3S)

This is the core "attention" mechanism. Instead of trying to pick the perfect 6-DoF pose in one giant leap, the robot follows a fixed sequence of six "gazes":

  • Steps 1-2: Select a coarse, then fine 3D position.
  • Steps 3-5: Iteratively refine the orientation (rotation around Z, then Y, then Z again).
  • Step 6: Select a final position offset (critical for precise placing).

At each step, the robot "zooms in," producing a high-resolution height map of a smaller volume. This turns an exponential search problem into a logarithmic one. The system is trained using Deep RL (Gradient Monte Carlo) with a set of independent Convolutional Neural Networks (CNNs)—one for each step in the hierarchy.

Hierarchical sequence of gazes narrowing the search space through zoomed-in height maps.

Experimental Setup

Training was conducted in the OpenRAVE simulator using objects from 3DNet (specifically bottles and mugs) and primitive blocks. The system was then evaluated on a UR5 robot with a Robotiq gripper and a Structure depth sensor. The real-world test sets included 15 diverse wooden blocks, 6 different mugs, and 7 different bottles with 3 coasters. The authors evaluated three specific tasks: stacking blocks, placing mugs upright on a table, and placing bottles upright on coasters.

Key Results and Insights

  • Superior Grasping: In simulation, the HSE3S method achieved a 98% success rate on block grasping, significantly outperforming the popular Grasp Pose Detection (GPD) algorithm, which achieved only 51% under the same conditions.
  • Robust Transfer: On the real UR5 robot, the system maintained high performance without additional fine-tuning. It achieved a 96% grasp success for blocks, 86% for mugs, and 90% for bottles.
  • Attention as a Constraint: The most powerful idea here is that "attention" is a mathematical constraint. By forcing the robot to look at smaller and smaller volumes, the amount of data the agent needs to process to make a precise decision is drastically reduced.
  • Joint Learning: The robot doesn't just learn to grasp; it learns to grasp in a way that makes the subsequent placement possible. The reward for a successful placement is tied to the success of the grasp, forcing the agent to discover functional grasps.

Limitations and Future Work

Real-world failures were often caused by "noise" in the coordinate transform between the camera and the robot base. If the point cloud is slightly misaligned, precise placement (like stacking a small block) fails. Furthermore, the current work focuses on a single pick-and-place sequence. Future work should extend this to longer-horizon tasks, such as building a tall tower of blocks, and explore "Actor-Critic" methods to output optimal 6-DoF poses directly, potentially speeding up execution.

Summary updated

BibTeX
@inproceedings{gualtieri2018learning,
  author = {Marcus Gualtieri and Robert Platt Jr.},
  title = {{Learning 6-DoF Grasping and Pick-Place Using Attention Focus}},
  booktitle = {Conference on Robot Learning},
  year = {2018},
}
Figure