A Mobile Manipulation System for One-Shot Teaching of Complex Tasks in Homes

Max Bajracharya, James Borders, Daniel M. Helmick, Thomas Kollar, Michael Laskey, John Leichty, Jeremy Ma, Umashankar Nagarajan, Akiyoshi Ochiai, Josh Petersen, Krishna Shankar, Kevin Stone, Yutaka Takaoka

Abstract

We describe a mobile manipulation hardware and software system capable of autonomously performing complex human-level tasks in real homes, after being taught the task with a single demonstration from a person in virtual reality. This is enabled by a highly capable mobile manipulation robot, whole-body task space hybrid position/force control, teaching of parameterized primitives linked to a robust learned dense visual embeddings representation of the scene, and a task graph of the taught behaviors. We demonstrate the robustness of the approach by presenting results for performing a variety of tasks, under different environmental conditions, in multiple real homes. Our approach achieves 85% overall success rate on three tasks that consist of an average of 45 behaviors each. The video is available at: https://youtu.be/HSyAGMGikLk.
BibTeX
@inproceedings{bajracharya2020mobile,
  author = {Max Bajracharya and James Borders and Daniel M. Helmick and Thomas Kollar and Michael Laskey and John Leichty and Jeremy Ma and Umashankar Nagarajan and Akiyoshi Ochiai and Josh Petersen and Krishna Shankar and Kevin Stone and Yutaka Takaoka},
  title = {{A Mobile Manipulation System for One-Shot Teaching of Complex Tasks in Homes}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2020},
  doi = {10.1109/icra40945.2020.9196677},
}

Loading related content...

Figure