An Efficient Image-to-Image Translation HourGlass-based Architecture for Object Pushing Policy Learning

Marco Ewerton, Ángel Martínez-González, Jean-Marc Odobez

Abstract

Robotic manipulation tasks, such as pushing objects into a target area, require high spatial precision and an understanding of global context. Traditional reinforcement learning (RL) approaches often rely on image classification backbones (e.g., DenseNet) that downsample inputs, losing the fine-grained resolution necessary for precise action selection. This paper proposes an architectural shift, framing robotic pushing as an image-to-image translation problem. By utilizing an Hourglass (U-Net style) architecture and decoupling valid push detection from goal-oriented reward prediction, the proposed system achieves significantly higher success rates and better spatial precision than existing state-of-the-art baselines.

Introduction

In robotic manipulation, "pushing" is a foundational skill used for clearing paths or organizing objects. However, teaching a robot exactly where and how to push remains a difficult problem in RL. Most current deep RL approaches for manipulation rely on backbone architectures designed for image classification. While effective for identifying objects, these networks progressively downsample images to extract high-level features, destroying the spatial resolution needed to decide which specific pixel should initiate a push.

Furthermore, standard convolutional networks are translation invariant, meaning they treat a push the same way regardless of its location on a table. In tasks like "push objects into a box," the location relative to the goal is critical. This research addresses these limitations by introducing the PushInBox architecture.

Methodology

The PushInBox architecture treats pushing as a dense prediction task, utilizing several key components to maintain spatial awareness and improve learning efficiency.

The Hourglass Backbone

Instead of a standard CNN, the researchers employ an Hourglass (U-Net) architecture. This allows the network to capture global context (the entire workspace) while maintaining high-resolution spatial information at the output layer, which is essential for precise manipulation.

The experimental setup featuring a UR5 robot arm and the simulated environment used for training and evaluation.

Decoupled Subnetworks and Action Space

The system splits the task into two specialized components:

  1. PushMask: A binary classifier that predicts if a push at a specific pixel and orientation will cause any change in the environment, effectively filtering out "pushes in the void."
  2. PushReward: A network that predicts the expected utility (Q-value) of a push for the specific task of moving objects into the box.

To handle orientation, the robot considers 16 different pushing angles. The system rotates the input image 16 times, passing each through the network to generate a dense "utility map" for every possible action.

The general pipeline for dense prediction of pushing rewards, illustrating how rotated heightmaps are processed to find the optimal push.

Positional Encoding

To overcome the limitations of translation invariance, the network is fed 2D coordinates representing each pixel's position relative to the target box. This allows the model to learn position-dependent behaviors, such as pushing toward the center when an object is at the edge of the workspace.

Experimental Setup

The study was conducted in a simulated environment using a UR5 robot arm and a depth camera. The researchers utilized three primary data sources:

  • Heuristic Dataset ($D_m$): Used to pre-train the PushMask network by identifying object edges.
  • Offline RL Dataset ($D_{rew}$): A collection of random and masked pushes used to initialize the PushReward network.
  • Evaluation Benchmark: 100 random scenarios containing 10 distinct objects with unknown physical properties like friction and mass.

Key Results

The Hourglass-based models consistently outperformed DenseNet-based baselines in both success rate and efficiency:

  • Success Rate: The best Hourglass model (HGLASSN with masking) successfully pushed an average of 8.96 out of 10 objects into the box, compared to only 5.38 for the DenseNet baseline.
  • Efficiency: The proposed architecture achieved a higher "objects-per-action" ratio (0.42 vs. 0.25), indicating more purposeful movements.
  • Precision: Qualitative analysis showed that Hourglass utility maps are significantly sharper and more localized than the "blurry" maps produced by DenseNet, leading to more precise robot movements.

Conclusion

By framing robotic pushing as an image-to-image translation task, this research demonstrates that maintaining spatial resolution is vital for manipulation. The Hourglass architecture, combined with action masking and positional encoding, provides a robust framework for learning complex pushing policies. Future work will focus on bridging the simulation-to-reality gap and handling dynamic objects with more complex physical behaviors.

Summary updated

BibTeX
@inproceedings{ewerton2021efficient,
  author = {Marco Ewerton and Ángel Martínez-González and Jean-Marc Odobez},
  title = {{An Efficient Image-to-Image Translation HourGlass-based Architecture for Object Pushing Policy Learning}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2021},
  doi = {10.1109/iros51168.2021.9636601},
}
Figure