This paper, presented at RSS 2022, introduces a framework for optimizing how human-robot teams tackle a sequence of tasks by strategically deciding when a robot should work autonomously, when it should ask for help, and—crucially—when it should be taught a new skill.
TL;DR
The "Act, Delegate, or Learn" (ADL) framework treats robot learning as a long-term investment. By using Mixed Integer Programming and a novel "precondition prediction" model, the system decides whether to perform a task, hand it off to a human, or invest in a demonstration to gain a skill that pays off for future tasks.
What Problem Does This Paper Address?
In environments like agile manufacturing or space exploration, robots face a diverse set of tasks. Achieving full autonomy is often too expensive or time-consuming. Traditionally, researchers have looked at two separate solutions:
- Task Allocation: Deciding which existing skills the robot should use vs. what the human should do.
- Learning from Demonstration (LfD): Teaching the robot new skills.
The gap lies in the synergy between these two. If a robot encounters a difficult task, should it simply delegate it to a human (low immediate cost) or ask for a demonstration (high immediate cost, but potentially high future reward)? Existing planners are often "myopic"—they only look at the current task—or they assume the robot’s capabilities are static. This paper proposes a way to reason about the long-term "return on investment" of teaching a robot a new skill.
Methods
The authors formulate the problem as a Stochastic Shortest Path (SSP) problem, aiming to minimize the total expected effort (human time + robot time + failure costs).
- The ADL Framework: For every task in a sequence, the planner chooses between three actions:
- Act ($a_{rob}$): Robot attempts the task with its current library.
- Delegate ($a_{hum}$): Human performs the task; the robot learns nothing.
- Learn ($a_{demo}$): Human demonstrates the task; the robot completes the task and adds a new skill to its library.
- Mixed Integer Programming (MIP): Standard graph searches (like A*) explode exponentially as the number of tasks increases. The authors transform the planning problem into a MIP, allowing off-the-shelf solvers like Gurobi to find optimal schedules for 20+ tasks in under a second.
- Precondition Prediction Model: This is the "secret sauce." To decide if learning a skill is worth it, the robot needs to "foresee" if a skill learned for Task A will work for Task B. They train a neural network classifier that takes two task descriptions and predicts the probability of success if a skill is transferred between them.
Datasets
The framework was evaluated on two distinct domains:
- Peg-in-Hole (Simulation): 20 tasks involving inserting blocks into slots under uncertainty, simulated in Nvidia Isaac Gym. This was used to benchmark the planner against myopic baselines.
- Lego Stacking (Real-World): A set of 15 unique Lego-style parts. The robot (a Franka Emika Panda) had to stack these onto a base plate.
- Novelty: The precondition model for the Lego task was trained in a "low-fidelity" abstract 2D simulation to save time, then applied to the real 3D robot.
Key Insights and Ideas
- Learning as a Capital Investment: The core "aha moment" is treating a human demonstration not just as a way to finish a task, but as a "facility" that, once opened, reduces the cost of all future similar tasks.
- Sim-to-Real for Relationships, Not Physics: While transferring high-speed manipulation skills from simulation to reality is hard, the authors realized that relationships between tasks (e.g., "if I can stack a 2x2 brick, I can stack a 2x4 brick") are much easier to learn in simulation and transfer to the real world.
- Linearization of the "Max" Term: The objective function involves a
maxover all available skills to find the best one for a task. The authors show how to linearize this for the MIP solver, making a complex non-linear problem solvable with high efficiency.
Key Results
- Strict Superiority: In the peg-insertion tests, ADL consistently achieved lower total costs than baselines like Confidence-Based Autonomy (which only asks for help when unsure) or Act-Learn Myopic (which doesn't look ahead).
- Optimal Synergy: In the Lego stacking experiments, the ADL planner was the only one that successfully balanced all three options. It recognized when the skill library was empty (delegating/learning) and when it had become "smart" enough to take over (acting).
- Efficiency: The MIP formulation allowed the system to solve 20-task sequences optimally in less than a second, whereas traditional search methods would be intractable.
Limitations and Future Work
- Fixed Sequences: Currently, the model assumes tasks arrive in a fixed order. In many real-world scenarios, a robot could choose the order of tasks to maximize learning efficiency.
- One Skill, One Task: The precondition model assumes a skill is trained on exactly one task. Future work could explore "generalized" skills trained on multiple demonstrations.
- Data Hunger: Training the precondition model still requires a significant amount of simulated data. Making this process more sample-efficient is a primary goal for future iterations.
Summary updated