Synergistic Scheduling of Learning and Allocation of Tasks in Human-Robot Teams

Shivam Vats, Oliver Kroemer, Maxim Likhachev

This paper, presented at RSS 2022, introduces a framework for optimizing how human-robot teams tackle a sequence of tasks by strategically deciding when a robot should work autonomously, when it should ask for help, and—crucially—when it should be taught a new skill.

TL;DR

The "Act, Delegate, or Learn" (ADL) framework treats robot learning as a long-term investment. By using Mixed Integer Programming and a novel "precondition prediction" model, the system decides whether to perform a task, hand it off to a human, or invest in a demonstration to gain a skill that pays off for future tasks.

What Problem Does This Paper Address?

In environments like agile manufacturing or space exploration, robots face a diverse set of tasks. Achieving full autonomy is often too expensive or time-consuming. Traditionally, researchers have looked at two separate solutions:

  1. Task Allocation: Deciding which existing skills the robot should use vs. what the human should do.
  2. Learning from Demonstration (LfD): Teaching the robot new skills.

The gap lies in the synergy between these two. If a robot encounters a difficult task, should it simply delegate it to a human (low immediate cost) or ask for a demonstration (high immediate cost, but potentially high future reward)? Existing planners are often "myopic"—they only look at the current task—or they assume the robot’s capabilities are static. This paper proposes a way to reason about the long-term "return on investment" of teaching a robot a new skill.

Methods

The authors formulate the problem as a Stochastic Shortest Path (SSP) problem, aiming to minimize the total expected effort (human time + robot time + failure costs).

  • The ADL Framework: For every task in a sequence, the planner chooses between three actions:
  • Act ($a_{rob}$): Robot attempts the task with its current library.
  • Delegate ($a_{hum}$): Human performs the task; the robot learns nothing.
  • Learn ($a_{demo}$): Human demonstrates the task; the robot completes the task and adds a new skill to its library.
  • Mixed Integer Programming (MIP): Standard graph searches (like A*) explode exponentially as the number of tasks increases. The authors transform the planning problem into a MIP, allowing off-the-shelf solvers like Gurobi to find optimal schedules for 20+ tasks in under a second.
  • Precondition Prediction Model: This is the "secret sauce." To decide if learning a skill is worth it, the robot needs to "foresee" if a skill learned for Task A will work for Task B. They train a neural network classifier that takes two task descriptions and predicts the probability of success if a skill is transferred between them.

Datasets

The framework was evaluated on two distinct domains:

  1. Peg-in-Hole (Simulation): 20 tasks involving inserting blocks into slots under uncertainty, simulated in Nvidia Isaac Gym. This was used to benchmark the planner against myopic baselines.
  2. Lego Stacking (Real-World): A set of 15 unique Lego-style parts. The robot (a Franka Emika Panda) had to stack these onto a base plate.
  3. Novelty: The precondition model for the Lego task was trained in a "low-fidelity" abstract 2D simulation to save time, then applied to the real 3D robot.

Key Insights and Ideas

  • Learning as a Capital Investment: The core "aha moment" is treating a human demonstration not just as a way to finish a task, but as a "facility" that, once opened, reduces the cost of all future similar tasks.
  • Sim-to-Real for Relationships, Not Physics: While transferring high-speed manipulation skills from simulation to reality is hard, the authors realized that relationships between tasks (e.g., "if I can stack a 2x2 brick, I can stack a 2x4 brick") are much easier to learn in simulation and transfer to the real world.
  • Linearization of the "Max" Term: The objective function involves a max over all available skills to find the best one for a task. The authors show how to linearize this for the MIP solver, making a complex non-linear problem solvable with high efficiency.

Key Results

  • Strict Superiority: In the peg-insertion tests, ADL consistently achieved lower total costs than baselines like Confidence-Based Autonomy (which only asks for help when unsure) or Act-Learn Myopic (which doesn't look ahead).
  • Optimal Synergy: In the Lego stacking experiments, the ADL planner was the only one that successfully balanced all three options. It recognized when the skill library was empty (delegating/learning) and when it had become "smart" enough to take over (acting).
  • Efficiency: The MIP formulation allowed the system to solve 20-task sequences optimally in less than a second, whereas traditional search methods would be intractable.

Limitations and Future Work

  • Fixed Sequences: Currently, the model assumes tasks arrive in a fixed order. In many real-world scenarios, a robot could choose the order of tasks to maximize learning efficiency.
  • One Skill, One Task: The precondition model assumes a skill is trained on exactly one task. Future work could explore "generalized" skills trained on multiple demonstrations.
  • Data Hunger: Training the precondition model still requires a significant amount of simulated data. Making this process more sample-efficient is a primary goal for future iterations.

Summary updated

Abstract

We consider the problem of completing a set of $n$ tasks with a human-robot team using minimum effort. In many domains, teaching a robot to be fully autonomous can be counterproductive if there are finitely many tasks to be done. Rather, the optimal strategy is to weigh the cost of teaching a robot and its benefit- how many new tasks it allows the robot to solve autonomously. We formulate this as a planning problem where the goal is to decide what tasks the robot should do autonomously (act), what tasks should be delegated to a human (delegate) and what tasks the robot should be taught (learn) so as to complete all the given tasks with minimum effort. This planning problem results in a search tree that grows expo-nentially with $n$ - making standard graph search algorithms intractable. We address this by converting the problem into a mixed integer program that can be solved efficiently using off-the-shelf solvers with bounds on solution quality. To predict the benefit of learning, we use an approximate simulation model of the tasks to train a precondition model that is parameterized by the training task. Finally, we evaluate our approach on peg insertion and Lego stacking tasks- both in simulation and real-world, showing substantial savings in human effort.
BibTeX
@inproceedings{vats2022synergistic,
  author = {Shivam Vats and Oliver Kroemer and Maxim Likhachev},
  title = {{Synergistic Scheduling of Learning and Allocation of Tasks in Human-Robot Teams}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2022},
  url = {https://openreview.net/forum?id=UPvQ881CKS},
}

Loading related content...

Figure