Imitation of Manipulation Skills Using Multiple Geometries

Boyang Ti, Yongsheng Gao, Jie Zhao, Sylvain Calinon

Abstract

Traditional robot Learning from Demonstration (LfD) typically relies on Cartesian coordinate systems, which often fail to capture the inherent geometric logic of many manipulation tasks. This paper introduces a novel framework that automatically identifies the most natural geometric representation—such as cylindrical or spherical coordinate systems—to encode a skill. By leveraging Riemannian manifold statistics to analyze user demonstrations, the system extracts task invariants that significantly improve generalization in tasks like grasping and hinge-based interactions.

Introduction

When teaching robots through demonstration, movements are usually recorded in standard Cartesian ($x, y, z$) coordinates. However, many daily tasks possess inherent geometric structures that Cartesian systems represent poorly. For example, opening a door is fundamentally a rotation around an axis (best described by cylindrical coordinates), while reaching for a lightbulb may be more consistently described by angles (spherical coordinates).

If a robot learns only in Cartesian space, it may fail to distinguish essential task invariants from noise. This leads to poor generalization: if an object's position changes slightly, the learned path may no longer be valid because the underlying geometric "logic" was not captured.

Methodology

The proposed framework treats different coordinate systems as a "dictionary" of candidate representations, focusing on three primary geometries: Prismatic (Cartesian), Cylindrical, and Spherical. Because these are not all "flat" Euclidean spaces, the authors utilize Riemannian manifolds to handle the underlying mathematics.

Geometric primitives in daily objects and tasks, showing how different shapes suggest different coordinate systems.

Statistical Extraction and Selection

The system observes human demonstrations and calculates the variability of the movement across all candidate coordinate systems using Gaussian distributions extended to manifolds. The robot then automatically selects the coordinate system where the demonstrations are most consistent (i.e., have the lowest variance). This "winner-takes-all" approach assumes that the geometry with the highest consistency defines the "essence" of the task.

Optimal Control

To reproduce the skill, the authors utilize an iterative Linear Quadratic Regulator (iLQR). The cost function for this controller is derived directly from the learned Gaussian distributions in the selected coordinate systems, ensuring the robot adheres to the identified task invariants.

Experimental Evaluation

The approach was evaluated using both simulated environments (a 3-DOF planar robot) and a real-world 7-axis Franka Emika manipulator. The task suite included grasping six distinct objects—a chips can, a baseball, a bowl, a Rubik’s cube, a cracker box, and a pitcher handle—as well as a box-opening task.

Results and Discussion

The "Multiple Geometries" approach significantly outperformed the Cartesian baseline in generalization tests. For cylindrical objects like the chips can and bowl, the Cartesian baseline failed completely (0% success) when starting positions were randomized, while the proposed method achieved success rates of 84% and 90%, respectively.

Opening a door modeled as a rotation around a cylindrical axis, capturing the task's geometric invariant.

In the box-opening task, the robot successfully maintained the circular arc required by the hinge. This demonstrates the power of invariance: by focusing on the coordinate system where the data is most regular, the robot naturally ignores irrelevant perturbations, such as the exact $x,y,z$ path, in favor of maintaining a constant radius from the hinge. The system proved it could learn complex, non-linear constraints from as few as six demonstrations for grasping and even a single demonstration for box opening.

Conclusion

By providing robots with a "vocabulary" of geometries, this framework allows for simpler and more robust descriptions of complex actions. While current work relies on visual markers and a winner-takes-all selection, future iterations aim to integrate computer vision for automatic primitive detection and explore the fusion of multiple manifolds for even more nuanced behavior.

Summary updated

BibTeX
@inproceedings{ti2022imitation,
  author = {Boyang Ti and Yongsheng Gao and Jie Zhao and Sylvain Calinon},
  title = {{Imitation of Manipulation Skills Using Multiple Geometries}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2022},
  doi = {10.1109/iros47612.2022.9981683},
}
Figure