Abstract
Robotic manipulation tasks, such as pushing objects into a target area, require high spatial precision and an understanding of global context. Traditional reinforcement learning (RL) approaches often rely on image classification backbones (e.g., DenseNet) that downsample inputs, losing the fine-grained resolution necessary for precise action selection. This paper proposes an architectural shift, framing robotic pushing as an image-to-image translation problem. By utilizing an Hourglass (U-Net style) architecture and decoupling valid push detection from goal-oriented reward prediction, the proposed system achieves significantly higher success rates and better spatial precision than existing state-of-the-art baselines.
Introduction
In robotic manipulation, "pushing" is a foundational skill used for clearing paths or organizing objects. However, teaching a robot exactly where and how to push remains a difficult problem in RL. Most current deep RL approaches for manipulation rely on backbone architectures designed for image classification. While effective for identifying objects, these networks progressively downsample images to extract high-level features, destroying the spatial resolution needed to decide which specific pixel should initiate a push.
Furthermore, standard convolutional networks are translation invariant, meaning they treat a push the same way regardless of its location on a table. In tasks like "push objects into a box," the location relative to the goal is critical. This research addresses these limitations by introducing the PushInBox architecture.
Methodology
The PushInBox architecture treats pushing as a dense prediction task, utilizing several key components to maintain spatial awareness and improve learning efficiency.
The Hourglass Backbone
Instead of a standard CNN, the researchers employ an Hourglass (U-Net) architecture. This allows the network to capture global context (the entire workspace) while maintaining high-resolution spatial information at the output layer, which is essential for precise manipulation.
Decoupled Subnetworks and Action Space
The system splits the task into two specialized components:
- PushMask: A binary classifier that predicts if a push at a specific pixel and orientation will cause any change in the environment, effectively filtering out "pushes in the void."
- PushReward: A network that predicts the expected utility (Q-value) of a push for the specific task of moving objects into the box.
To handle orientation, the robot considers 16 different pushing angles. The system rotates the input image 16 times, passing each through the network to generate a dense "utility map" for every possible action.
Positional Encoding
To overcome the limitations of translation invariance, the network is fed 2D coordinates representing each pixel's position relative to the target box. This allows the model to learn position-dependent behaviors, such as pushing toward the center when an object is at the edge of the workspace.
Experimental Setup
The study was conducted in a simulated environment using a UR5 robot arm and a depth camera. The researchers utilized three primary data sources:
- Heuristic Dataset ($D_m$): Used to pre-train the
PushMasknetwork by identifying object edges. - Offline RL Dataset ($D_{rew}$): A collection of random and masked pushes used to initialize the
PushRewardnetwork. - Evaluation Benchmark: 100 random scenarios containing 10 distinct objects with unknown physical properties like friction and mass.
Key Results
The Hourglass-based models consistently outperformed DenseNet-based baselines in both success rate and efficiency:
- Success Rate: The best Hourglass model (HGLASSN with masking) successfully pushed an average of 8.96 out of 10 objects into the box, compared to only 5.38 for the DenseNet baseline.
- Efficiency: The proposed architecture achieved a higher "objects-per-action" ratio (0.42 vs. 0.25), indicating more purposeful movements.
- Precision: Qualitative analysis showed that Hourglass utility maps are significantly sharper and more localized than the "blurry" maps produced by DenseNet, leading to more precise robot movements.
Conclusion
By framing robotic pushing as an image-to-image translation task, this research demonstrates that maintaining spatial resolution is vital for manipulation. The Hourglass architecture, combined with action masking and positional encoding, provides a robust framework for learning complex pushing policies. Future work will focus on bridging the simulation-to-reality gap and handling dynamic objects with more complex physical behaviors.
Summary updated