A comparison of visual representations for real-world reinforcement learning in the context of vacuum gripping

Nico Sutter, Valentin N. Hartmann, Stelian CorosarXiv 2025 · DOI · arXiv

Introduction

Vacuum gripping is a staple of industrial automation, yet achieving a reliable seal in unstructured environments remains a significant challenge. Factors such as surface texture, deformable materials, and geometric irregularities like zip-ties or holes often cause traditional systems to fail. Researchers from ETH Zurich recently investigated how different visual representations—RGB, depth, and 3D voxel grids—impact the performance and generalization of reinforcement learning (RL) agents tasked with "last-inch" vacuum manipulation.

Methodology

The study utilized the SERL (Sample-Efficient robotic Reinforcement Learning) framework to train a 6-DoF UR5e robot arm equipped with a Robotiq EPick vacuum gripper. The sensing suite included two wrist-mounted Intel RealSense D405 cameras to provide local visual feedback.

The experimental setup alongside its corresponding 3D voxel grid representation, which captures the local geometry essential for a vacuum seal.

The researchers compared three primary visual encoders:

  1. RGB Images: Processed via a pre-trained ResNet18 backbone.
  2. Depth Images: Processed through a 2D convolutional neural network (CNN).
  3. Voxel Grids: Point clouds from the cameras were converted into a 3D grid of voxels and analyzed using a 3D CNN (VoxNet).

To maximize training efficiency, the team employed Observation Space Symmetries. By rotating observations into a canonical quadrant, they effectively quadrupled the available training data, allowing robust policies to be trained in under 90 minutes on a single consumer-grade GPU.

The logic flow of the RL policy, illustrating the transition from movement to force detection and suction verification.

Results and Generalization

The core finding of the research is that 3D voxel grids significantly outperform 2D representations, particularly when generalizing to unseen objects. While RGB-based agents often overfit to surface appearance (color and labels), voxel-based agents focus on the underlying geometry required for a successful seal.

Representation Success Rate (Seen) Success Rate (Unseen) Avg. Time to Grasp
Behavior Tree (Baseline) 86.7% 86.7% 5.32s
RGB (ResNet18) 96.7% 80.0% 5.28s
Voxel Grid (VoxNet) 100% 96.7% 4.34s

The voxel-based policy achieved a near-perfect success rate on novel boxes and was faster than both the RGB agent and the heuristic behavior tree baseline. This suggests that spatial awareness is more critical than visual detail for this specific manipulation task.

Conclusion

This research demonstrates that for contact-rich tasks like vacuum gripping, geometric representations are superior to appearance-based ones. By leveraging 3D voxel grids and symmetry-based data augmentation, robots can learn complex manipulation policies directly in the real world with high sample efficiency and exceptional generalization capabilities. Future work may explore more advanced 3D architectures like Point Transformers to further refine these capabilities in cluttered or dynamic environments.

Paper Citation: Nico Sutter, Valentin N. Hartmann, Stelian Coros. "A comparison of visual representations for real-world reinforcement learning in the context of vacuum gripping." arXiv:2503.02405v1 [cs.RO], 2025.

Summary updated

Abstract

When manipulating objects in the real world, we need reactive feedback policies that take into account sensor information to inform decisions. This study aims to determine how different encoders can be used in a reinforcement learning (RL) framework to interpret the spatial environment in the local surroundings of a robot arm. Our investigation focuses on comparing real-world vision with 3D scene inputs, exploring new architectures in the process. We built on the SERL framework, providing us with a sample efficient and stable RL foundation we could build upon, while keeping training times minimal. The results of this study indicate that spatial information helps to significantly outperform the visual counterpart, tested on a box picking task with a vacuum gripper. The code and videos of the evaluations are available at https://github.com/nisutte/voxel-serl.
BibTeX
@misc{sutter2025comparison,
  author = {Nico Sutter and Valentin N. Hartmann and Stelian Coros},
  title = {{A comparison of visual representations for real-world reinforcement learning in the context of vacuum gripping}},
  year = {2025},
  eprint = {2503.02405},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2503.02405},
}
Figure