Interactive Trajectory Adaptation through Force-guided Bayesian Optimization

Leonel Dario Rozo

TL;DR

This paper presents a novel framework for robots to adapt their learned task trajectories in real-time based on physical human guidance. By combining Learning from Demonstration (LfD) with force-guided Bayesian Optimization, robots can intuitively learn and incorporate human-intended trajectory modifications, making them more flexible in dynamic manufacturing environments.

What Problem Does This Paper Address?

In modern manufacturing, especially in Small and Medium-sized Enterprises (SMEs), robots need to be highly adaptable to changing environments and human collaborators. While robots can learn tasks from human demonstrations (LfD), these learned "nominal plans" often need to be adjusted on the fly to accommodate new requirements or unexpected situations. When these adjustments involve modifying the robot's end-effector trajectory, a human operator might want to physically guide the robot. The challenge lies in enabling the robot to accurately interpret noisy haptic feedback from human interaction, quickly adapt its trajectory, and seamlessly revert to the original plan when no further adaptation is needed. This paper tackles this by developing a system that can learn from demonstrations and then adapt based on physical human input.

Methods

The core of the proposed framework lies in two main components:

  • Learning a Nominal Task Plan with Hidden Semi-Markov Models (HSMM): The robot first learns a probabilistic model of the task from human demonstrations. An HSMM is used to capture both the observed sensorimotor patterns (like position, velocity, acceleration) and the temporal structure of the task, including state durations. This HSMM represents the robot's "nominal plan."
  • Force-Guided Bayesian Optimization for Adaptation: When a human physically interacts with the robot, their forces are sensed. These forces are interpreted as signals indicating a desired trajectory adaptation. Bayesian Optimization (BayesOpt) is employed to efficiently find the optimal local adjustments to the HSMM parameters (specifically, the means of the sensorimotor distributions) that minimize the disagreement forces between the robot and the human. This optimization is made data-efficient by restricting the search to local regions within the HSMM, guided by the identified current state.

Datasets

The experiments were conducted using synthetic demonstrations of 2D pick-and-place tasks, specifically C-shape and L-shape trajectories. These demonstrations included both 2D Cartesian trajectories and sensed force profiles. No new benchmarks were introduced; rather, existing task types relevant to manufacturing were simulated.

Key Insights and Ideas

  • Haptic Cues as Intent Signals: The paper highlights that physical interaction forces are a rich source of information for inferring human intentions regarding task adaptation.
  • Local Adaptation for Efficiency: Instead of re-optimizing the entire task plan, focusing on local adjustments within the HSMM (at the state level) significantly reduces the search space for Bayesian Optimization, making adaptation data-efficient.
  • Spatial-Temporal Adaptation: The framework can adapt both the spatial (position, velocity) and temporal (duration) aspects of the robot's trajectory, offering a more comprehensive adaptation capability.
  • Seamless Reversion to Nominal Plan: The local adaptation strategy allows the robot to return to its original nominal trajectory when human interaction ceases, ensuring task completion without permanent deviations.
  • Dimensionality Reduction is Crucial for BayesOpt: The authors found that attempting to optimize a high-dimensional parameter space (all HSMM means and covariances) led to convergence issues, underscoring the importance of their local, state-specific optimization approach.

Key Results

The framework successfully demonstrated data-efficient trajectory adaptations in simulated 2D pick-and-place tasks.

  • Quick Convergence: Bayesian Optimization converged to optimal local parameters in as few as 5-6 iterations for adapting specific HSMM states.
  • Force Minimization: The adapted trajectories resulted in disagreement forces close to zero, indicating successful alignment with human-imposed modifications.
  • Accurate Trajectory Deformation: The adapted trajectories closely matched the nominal plan when no external forces were applied and deformed as intended when human forces were introduced, without disrupting critical task phases like grasping or releasing.
  • Modification of Duration Probabilities: The adaptation process also successfully modified state duration probabilities, allowing the robot to spend more or less time in certain states as needed.

Limitations and Future Work

  • Covariance Adaptation: The current framework omits the adaptation of covariance matrices for HSMM states, which is acknowledged as a limitation due to the high dimensionality and positive-definiteness constraints involved. Future work aims to explore methods like Cholesky decomposition or exploiting the geometry of symmetric positive definite matrices for covariance adaptation.
  • Real-World Scenarios: The experiments were conducted in simulation. Future work will involve evaluating the framework in real-world scenarios, particularly those with time-varying force profiles.
  • Latent Space Representations: The authors plan to investigate how latent space representations of HSMM parameters can be used to alleviate the high-dimensional problem of covariance adaptation.

Summary updated

Abstract

Flexible manufacturing processes demand robots to easily adapt to changes in the environment and interact with humans. In such dynamic scenarios, robotic tasks may be programmed through learning-from-demonstration (LfD) approaches, where a nominal plan of the task is learned by the robot. However, the learned plan may need to be adapted in order to fulfill additional requirements or overcome unexpected environment changes. When the required adaptation occurs at the end-effector trajectory level, a human operator may want to intuitively show the robot the desired changes by physically interacting with it. In this scenario, the robot needs to understand the human intended changes from noisy haptic data, quickly adapt accordingly and execute the nominal task plan when no further adaptation is needed. This paper addresses the aforementioned challenges by leveraging LfD and Bayesian optimization to endow the robot with data-efficient adaptation capabilities. Our approach exploits the sensed interaction forces to guide the robot adaptation, and speeds up the optimization process by defining local search spaces extracted from the learned task model. We show how our framework quickly adapts the learned spatial-temporal patterns of the task, leading to deformed trajectory distributions that are consistent with the nominal plan and the changes introduced by the human.
BibTeX
@inproceedings{rozo2019interactive,
  author = {Leonel Dario Rozo},
  title = {{Interactive Trajectory Adaptation through Force-guided Bayesian Optimization}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2019},
  url = {https://openreview.net/forum?id=kRnyIQoIyE},
}
Figure