Introduction
Robotic manipulation of flat objects, such as books, phones, or thin plates, presents a significant challenge for standard parallel-jaw grippers. When these objects lie horizontally on a flat surface, they often enter "ungraspable poses"—states where the object is thinner than the gripper's minimum stroke, making it impossible for the robot to get its fingers underneath the item.
While humans intuitively solve this by sliding an object to the edge of a table or using a wall for leverage, existing robotic solutions are often limited. "Push-to-wall" strategies require specific environmental features and frequently fail when objects have beveled or slanted edges that the gripper cannot catch. This paper introduces a more versatile push-to-edge strategy, utilizing a Deep Reinforcement Learning (DRL) framework to enable robots to manipulate objects into graspable positions regardless of edge geometry or the presence of walls.
The PaG (Push-and-Grasp) Framework
The researchers developed a model-free DRL framework named PaG (Push-and-Grasp). The system is designed to synergize the two distinct actions of pushing and grasping into a cohesive policy.
Visual Representation and Policy Optimization
To handle high-dimensional visual input, the system employs a Variational Autoencoder (VAE) pre-trained on 100,000 simulated images. This compresses RGB camera data into a compact 128-dimensional latent vector, which serves as the input for the policy.
The policy itself is optimized using Proximal Policy Optimization (PPO). A key innovation in this work is the use of a single Actor-Critic network with shared layers for both pushing and grasping. By sharing layers, the model learns the underlying structural similarities between the tasks—specifically the requirement for stable contact—allowing the robot to understand the trade-off between moving an object toward the edge and accidentally pushing it off entirely.
Reward Engineering
The reward function ($R$) is carefully constructed to guide the learning process through three components:
- Progress Reward ($R_p$): Encourages movement toward the table's edge.
- Grasping Reward ($R_g$): Provides a high incentive for a successful lift.
- Common Reward ($R_c$): Based on the distance between the gripper and the object's center of mass to ensure stable contact during the pushing phase.
Sim-to-Real Transfer
To bridge the "reality gap" between the Isaac Gym simulator and a physical UR5 robot, the authors utilized CycleGAN. This generative model translates real-world camera images into the simulated style the network was trained on. This approach allows the policy to be deployed on a real robot without the need for time-consuming manual fine-tuning in the physical environment.
Experimental Evaluation
The framework was trained on a dataset of 80 unique objects across 10 categories (circles, squares, hexagons, etc.) with varying sizes. To test the system's robustness, it was evaluated against:
- Beveled Objects: Shapes with 30° chamfers that typically defeat push-to-wall methods.
- Irregular Shapes: Letters and large-scale objects with non-overlapping centers of mass.
- Real-World Objects: 13 daily items including cereal boxes, books, and calculators.
Key Results and Insights
The PaG framework demonstrated significant improvements over multi-network baselines and traditional strategies:
- Efficiency: The single-network architecture converged 2.5 times faster than approaches using separate networks for pushing and grasping. This confirms that sharing features for contact-based tasks significantly improves data efficiency.
- Robustness: The model achieved success rates exceeding 80% on beveled objects and maintained high performance on irregular shapes where the center of mass was off-center.
- Real-World Superiority: In physical tests, PaG maintained success rates between 73% and 93%. In contrast, the "push-to-wall" baseline often dropped to a 0% success rate when faced with beveled edges or environments without a physical wall.
Conclusion and Future Work
The push-to-edge strategy provides a robust solution for picking up flat objects in unstructured environments. By treating pushing and grasping as a unified problem through a single DRL network, the system achieves high efficiency and successful sim-to-real transfer.
Future research will aim to address current limitations, such as the assumption of uniform object density and the challenges posed by extremely complex textures. Additionally, the authors plan to explore "cluttered" scenarios where multiple ungraspable objects are layered or stacked, requiring more sophisticated sequencing of actions.
Summary updated