Learning Generalizable Pivoting Skills

Xiang Zhang, Siddarth Jain, Baichuan Huang, Masayoshi Tomizuka, Diego Romeres

Abstract

Pivoting is a fundamental contact-rich manipulation skill that enables robots to reorient objects into favorable positions for grasping. However, traditional Reinforcement Learning (RL) approaches often struggle with generalization, producing policies that are overfitted to specific object geometries. This paper presents a framework that learns a "Unitary" pivoting policy on a single standard object and generalizes it to a wide variety of unseen objects through learned linear projections of state and action spaces. By leveraging depth-based feature extraction and policy-gradient-based adaptation, the system achieves robust zero-shot transfer from simulation to the real world, even for irregularly shaped objects.

Introduction

In robotic manipulation, pivoting—the act of rotating an object around a point of contact—serves as a critical "pre-grasp" skill. It allows a robot to manipulate objects that are too heavy to lift directly or are positioned in a way that prevents a secure grasp. Despite its utility, pivoting is notoriously difficult to model due to its "contact-rich" nature, involving complex hybrid dynamics such as slipping, rolling, and varying friction.

While Reinforcement Learning (RL) has shown promise in mastering these dynamics, it typically results in policies that fail when the object’s dimensions or shape change. Existing solutions like Meta-RL or Domain Randomization often require prohibitive amounts of data or fail to generalize to objects significantly different from the training distribution. This research explores a more efficient alternative: can a robot learn a skill on one "standard" object and simply scale that knowledge to fit others?

Methodology

The proposed framework utilizes a three-stage pipeline designed to decouple the core manipulation skill from the specific physical dimensions of the object.

1. The Unitary Policy

The foundation of the system is a "Unitary" policy trained using Soft Actor-Critic (SAC) in a MuJoCo simulation. This policy is optimized to pivot a single, standard 9x9x3 cm box. The state space for this policy includes the poses of the object and the gripper, along with 3D force/torque sensor data. By focusing on a single object, the policy masters the fundamental physics of pivoting without the noise of varying geometries.

2. Object Feature Learning

To adapt to new objects, the system must first perceive their physical characteristics. The researchers employ a ResNet-18 backbone to process top-down depth images. This network is trained via supervised learning to predict an object's class (e.g., box, cylinder, peg) and its physical dimensions. This process maps visual data into a low-dimensional "feature space" that encodes the essential kinematics of the object.

3. State and Action Projections

The core innovation of this work lies in the use of two linear transformation matrices, $T_s$ (State Projection) and $T_a$ (Action Projection).

  • $T_s$ adjusts the input state of a novel object so that it appears "unitary" to the pre-trained policy.
  • $T_a$ scales the policy’s output actions to suit the specific physical scale of the new object.

These projections are learned using Proximal Policy Optimization (PPO) to maximize trajectory returns across a diverse set of simulated objects. By freezing the original Unitary policy and only training these projection layers, the system achieves high data efficiency.

Experimental Results

Simulation Performance

The framework was tested against a synthetic dataset of 400 objects across four classes (boxes, circles, cylinders, and pegs). The proposed method achieved a 75% success rate (30/40 objects in a held-out test set). In comparison, state-of-the-art Meta-RL (PEARL) failed to successfully pivot any objects in the same multi-object test environment, highlighting the difficulty of the task for standard adaptation methods.

Real-World and Zero-Shot Transfer

The system demonstrated remarkable robustness during real-world evaluation. Using AprilTags for object tracking, the robot achieved a 100% success rate on almost all standard test objects. Most notably, the system succeeded on "out-of-distribution" (OOD) objects, such as a mustard bottle and a sanitizer bottle. These objects possess non-convex, irregular geometries that were never encountered during the simulation training phase. This suggests that the linear projection approach effectively captures the underlying kinematic relationships required for pivoting, regardless of surface complexity.

Discussion and Conclusion

The success of this "Unitary" approach suggests that while trajectories for different objects vary, they often share a similar underlying structure in the correct feature space. Linear transformations proved sufficient to bridge the gap between vastly different scales and shapes.

However, limitations remain. The current model focuses on kinematic properties derived from vision and does not explicitly account for dynamic properties like mass distribution or friction coefficients. Future research may explore integrating tactile feedback or non-linear state-dependent transformations to handle more complex manipulation tasks or heavier objects. Nonetheless, this work provides a powerful, data-efficient template for generalizable robotic skills in contact-rich environments.

Summary updated

BibTeX
@inproceedings{zhang2023learning,
  author = {Xiang Zhang and Siddarth Jain and Baichuan Huang and Masayoshi Tomizuka and Diego Romeres},
  title = {{Learning Generalizable Pivoting Skills}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2023},
  doi = {10.1109/icra48891.2023.10161271},
}
Figure