Beyond lowest-warping cost action selection in trajectory transfer

Dylan Hadfield-Menell, Alex X. Lee, Chelsea Finn, Eric Tzeng, Sandy H. Huang, Pieter Abbeel

Abstract

We consider the problem of learning from demonstrations to manipulate deformable objects. Recent work [1], [2], [3] has shown promising results that enable robotic manipulation of deformable objects through learning from demonstrations. Their approach is able to generalize from a single demonstration to new test situations, and suggests a nearest neighbor approach to select a demonstration to adapt to a given test situation. Such a nearest neighbor approach, however, ignores important aspects of the problem: brittleness (versus robustness) of demonstrations when generalized through this process, and the extent to which a demonstration makes progress towards a goal. In this paper, we frame the problem of selecting which demonstration to transfer as an options Markov decision process (MDP). We present max-margin Q-function estimation: an approach to learn a Q-function from expert demonstrations. Our learned policies account for variability in robustness of demonstrations and the sequential nature of our tasks. We developed two knot-tying benchmarks to experimentally validate the effectiveness of our proposed approach. The selection strategy described in [2] achieves success rates of 70% and 54%, respectively. Our approach performs significantly better, with success rates of 88% and 76%, respectively.
BibTeX
@inproceedings{hadfieldmenell2015beyond,
  author = {Dylan Hadfield-Menell and Alex X. Lee and Chelsea Finn and Eric Tzeng and Sandy H. Huang and Pieter Abbeel},
  title = {{Beyond lowest-warping cost action selection in trajectory transfer}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2015},
  doi = {10.1109/icra.2015.7139644},
}

Loading related content...

Figure