Active Incremental Learning of Robot Movement Primitives

Guilherme Maeda, Marco Ewerton, Takayuki Osa, Baptiste Busch, Jan Peters

Abstract

Robots that can learn over time by interacting with non-technical users must be capable of acquiring new motor skills, incrementally. The problem then is deciding when to teach the robot a new skill or when to refine an existing one. This paper introduces a framework that allows robots to "know what they don't know" by combining Gaussian Processes (GPs) with Dynamical Movement Primitives (DMPs). The system enables a robot to quantify its confidence in a motor skill and actively request a human demonstration only when its uncertainty exceeds a safe threshold.

Introduction

In Imitation Learning (or Learning from Demonstration), robots typically rely on a human teacher to decide when a new skill needs to be taught or an old one corrected. This paradigm is limited by two factors: it requires constant human supervision, and it risks "hidden incompetence," where a robot might attempt a dangerous or failing trajectory because its internal model cannot accurately generalize to a new context. This research addresses the "when to learn" problem, enabling robots to proactively request assistance: "I'm not confident I can reach that object safely; can you show me how?"

Methodology

The authors propose a two-layered approach to movement generation that balances generalization with precision.

Gaussian Processes for Trajectory Prediction

Gaussian Processes (GPs) are used to map a "context" (the 3D goal position) to a full trajectory. The primary advantage of GPs in this context is their ability to provide a variance (uncertainty) measure. If a goal is far away from any previous demonstrations, the GP's uncertainty increases, signaling a lack of confidence.

Dynamical Movement Primitives for Execution

While GPs provide the "shape" of the move, Dynamical Movement Primitives (DMPs) act as the "engine." A DMP ensures the robot reaches the exact goal point, even if the GP's prediction is slightly off.

Comparison of GP extrapolation and DMP correction.

As illustrated in Figure 4, the DMP correction allows the robot to reach the target while the GP provides the underlying trajectory shape and uncertainty estimates (visualized as ellipses).

Experimental Evaluation

The framework was evaluated using a 7-degree-of-freedom (7-DOF) lightweight robot arm in several custom setups, most notably an assembly scenario involving 10 different objects.

The experimental setup for the assembly task.

Key Results

  • Minimal Demonstrations: In the assembly task (Figure 5), the robot achieved high confidence for all 10 objects using only 5 demonstrations.
  • Superiority to Random Sampling: The active learning approach reached confidence thresholds faster and with lower variance than uniform random requests.
  • Workspace Scaling: In "Increasing Coverage" experiments, the robot's "confident" reachable workspace grew 8x larger (from 0.036 m³ to 0.303 m³) after just 7 active requests.

Discussion

A key finding is the "neighbor effect": teaching the robot one skill (e.g., reaching a middle screw) significantly lowers its uncertainty for nearby tasks. By calculating uncertainty in Cartesian space (cm), the authors make the robot's confidence physically interpretable for the user. This "safety through hesitation" prevents the robot from attempting guesses likely to result in collisions.

Conclusion

This framework provides a robust method for incremental skill acquisition. While current limitations include manual uncertainty thresholds and simplified goal contexts, the integration of GPs and DMPs offers a clear path toward robots that can safely and efficiently learn from human interaction in complex environments.

Summary updated

Abstract

Robots that can learn over time by interacting with non-technical users must be capable of acquiring new motor skills, incrementally. The problem then is deciding when to teach the robot a new skil...
BibTeX
@inproceedings{maeda2017active,
  author = {Guilherme Maeda and Marco Ewerton and Takayuki Osa and Baptiste Busch and Jan Peters},
  title = {{Active Incremental Learning of Robot Movement Primitives}},
  booktitle = {Conference on Robot Learning},
  year = {2017},
  url = {https://openreview.net/forum?id=vW_QvYdH7q6},
}
Figure