Skill Acquisition via Automated Multi-Coordinate Cost Balancing

Harish Ravichandar, S. Reza Ahmadzadeh, M. Asif Rana, Sonia Chernova

TL;DR

Multi-Coordinate Cost Balancing (MCCB) is a new framework that allows robots to learn movement skills by analyzing demonstrations through three geometric "lenses" simultaneously: position, direction, and shape. By automatically learning how to weight these different perspectives, MCCB generates reproductions that are more accurate and adaptable than traditional methods that rely on a single coordinate system.

What Problem Does This Paper Address?

In the field of Learning from Demonstration (LfD), we want robots to watch a human perform a task and then replicate it. Traditionally, researchers have encoded these skills in a single coordinate system—usually Cartesian (standard X, Y, Z coordinates).

However, a single coordinate system is rarely enough to capture the "essence" of a skill. For example, if you are teaching a robot to write a letter, the specific shape (curvature) of the stroke might be more important than its absolute position on the table. Conversely, for a task like plugging in a cable, the absolute position is critical.

The challenge is that the "best" coordinate system varies from task to task, and manually tuning the importance of different geometric features is tedious and prone to error. This paper addresses the need for a unified way to capture multiple geometric properties and automatically determine which ones matter most for a specific skill.

Methods

The authors introduce MCCB, which operates in three stages:

  1. Multi-Coordinate Encoding: Instead of picking one system, MCCB transforms demonstrations into three "differential coordinates":
  2. Cartesian: Captures absolute position.
  3. Tangent: Captures the local direction of motion (1st-order derivative).
  4. Laplacian: Captures the local shape or curvature (2nd-order derivative).
  5. Statistical Modeling: The framework uses Gaussian Mixture Models (GMMs) to learn the joint probability of time and movement in each of these three spaces. Using Gaussian Mixture Regression (GMR), the robot can then estimate the expected movement and the associated variance (uncertainty) at any given time.
  6. Automated Cost Balancing: This is the core innovation. MCCB formulates a blended cost function where the robot tries to minimize deviations in all three coordinate systems at once. Crucially, it performs a meta-optimization to learn the optimal weights ($\alpha$) for these costs directly from the demonstration data. If the human was very consistent in "shape" but messy in "position," the algorithm automatically assigns a higher weight to the Laplacian coordinate.
  7. Convex Optimization: The final trajectory is generated by solving a constrained convex optimization problem, ensuring the robot hits specific start, end, or "via" points while maintaining the learned geometric features.

Datasets

The authors evaluated MCCB on four distinct datasets:

  • LASA Handwriting Library: A standard benchmark containing 25 different handwriting motions (e.g., "S" shapes, spirals), each with 7 demonstrations.
  • Picking Skill: Kinesthetic demonstrations of a robot picking up magnets from two different blocks.
  • Pressing Skill: Demonstrations of a robot pressing two cylindrical pegs into holes.
  • Pushing Skill: Demonstrations of a robot sliding the lid of a wooden box closed.

Key Insights and Ideas

  • No "One Size Fits All": The experiments proved that different tasks rely on different geometries. For the "Pushing" task, the Laplacian coordinate was the most relevant, whereas the "Pressing" task relied heavily on Cartesian coordinates.
  • Uniform Weighting is Not Enough: Simply averaging the three coordinate systems (uniform weighting) often performed worse than just picking the single best one. This highlights the necessity of the "Cost Balancing" meta-optimization to find the right ratio.
  • Variance as a Guide: By incorporating the inverse of the expected variance into the cost function, the robot naturally ignores "noise" in the demonstrations. If a human demonstrator varies wildly in their path but always hits a specific curve, the robot learns that the curve is the essential part of the skill.

Key Results

MCCB was compared against four baselines: Cartesian-only, Tangent-only, Laplacian-only, and Uniform Weighting.

  • Consistent Superiority: MCCB outperformed all baselines across all four datasets using metrics like Sum of Squared Errors (SSE) and Frechet Distance (which measures spatial similarity).
  • Handwriting: On the LASA dataset, MCCB successfully captured complex shapes that Cartesian-only methods struggled to replicate when start and end points were changed.
  • Complex Skills: In the "Pushing" task, MCCB achieved a Frechet Distance nearly 5x lower than the Cartesian baseline, demonstrating its ability to preserve the "feel" of a motion even under new constraints.

Limitations and Future Work

  • Temporal Alignment: The current method relies on Dynamic Time Warping (DTW) to align demonstrations of different lengths. While effective, this adds a preprocessing step that could be integrated more fluidly.
  • Model Agnostic but GMM-focused: While the authors state MCCB can work with any statistical model, the paper focuses exclusively on GMMs. Testing with Neural Networks or Gaussian Processes could be a future step.
  • Obstacle Avoidance: The current optimization handles point constraints (start/end/via) but does not explicitly incorporate obstacle avoidance or robot-specific constraints like joint limits (manipulability). The authors suggest this as a promising direction for future integration.

Summary updated

Abstract

We propose a learning framework, named Multi-Coordinate Cost Balancing (MCCB), to address the problem of acquiring point-to-point movement skills from demonstrations. MCCB encodes demonstrations simultaneously in multiple differential coordinates that specify local geometric properties. MCCB generates reproductions by solving a convex optimization problem with a multi-coordinate cost function and linear constraints on the reproductions, such as initial, target, and via points. Further, since the relative importance of each coordinate system in the cost function might be unknown for a given skill, MCCB learns optimal weighting factors that balance the cost function. We demonstrate the effectiveness of MCCB via detailed experiments conducted on one handwriting dataset and three complex skill datasets.
BibTeX
@inproceedings{ravichandar2019skill,
  author = {Harish Ravichandar and S. Reza Ahmadzadeh and M. Asif Rana and Sonia Chernova},
  title = {{Skill Acquisition via Automated Multi-Coordinate Cost Balancing}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2019},
  url = {https://openreview.net/forum?id=zKehLWcbIG},
}
Figure