Abstract
In many human-robot collaborative domains, achieving full robot autonomy is often counterproductive due to the high costs of training relative to the number of tasks. This paper addresses the challenge of completing a set of $n$ tasks with minimum effort by weighing the cost of teaching a robot against the long-term benefit of increased autonomy. We formulate this as a planning problem—deciding whether to Act (robot performs the task), Delegate (human performs the task), or Learn (human demonstrates the task for future use). To overcome the exponential complexity of standard graph searches, we transform the problem into a Mixed Integer Program (MIP) solvable with off-the-shelf tools. Furthermore, we introduce a precondition prediction classifier to estimate the transferability of learned skills. Our approach, validated in simulation and real-world Lego stacking tasks, demonstrates significant reductions in human effort compared to traditional myopic strategies.
Introduction
In environments such as agile manufacturing or space exploration, robots are frequently confronted with diverse and novel tasks. While Learning from Demonstration (LfD) allows robots to acquire new capabilities, teaching a robot every possible skill is often inefficient. Conversely, delegating every difficult task to a human fails to leverage the robot's potential for automation.
The central challenge lies in the synergy between task allocation and learning. Existing frameworks typically treat these as separate problems: planners either decide how to use existing skills or focus on how to acquire new ones. This paper introduces the "Act, Delegate, or Learn" (ADL) framework, which treats robot learning as a strategic long-term investment. By reasoning about the "return on investment" for each human demonstration, the system can decide when to incur a high immediate cost (teaching) to reap future rewards (autonomy).
The ADL Framework
The ADL framework models the task sequence as a planning problem where the goal is to minimize total expected effort, including human time, robot time, and the costs associated with potential failures. For every task in a sequence, the planner selects from three primary actions:
- Act ($a_{rob}$): The robot attempts the task using its current skill library. This is low-cost if successful but carries a risk of failure.
- Delegate ($a_{hum}$): A human performs the task. This is reliable but provides no long-term benefit to the robot's capabilities.
- Learn ($a_{demo}$): A human provides a demonstration. While this requires significant human effort upfront, the robot completes the task and adds a new skill to its library, potentially reducing the cost of similar future tasks.
Optimization via Mixed Integer Programming
Formulating this as a Stochastic Shortest Path (SSP) problem typically results in a search tree that grows exponentially with the number of tasks ($n$). To maintain tractability, the authors convert the planning problem into a Mixed Integer Program (MIP). By linearizing the objective function—specifically the "max" term used to select the best available skill for a given task—the framework can utilize solvers like Gurobi to find optimal schedules for sequences of 20+ tasks in less than a second.
Precondition Prediction
To evaluate the potential benefit of learning a new skill, the robot must predict whether a skill learned for Task A will successfully transfer to Task B. The authors developed a "precondition prediction" model—a neural network classifier that takes descriptions of two tasks and outputs the probability of success for a transferred skill. Interestingly, the authors found that while low-level physics are hard to transfer from simulation to reality, the relationships between tasks (e.g., "if I can stack a small brick, I can stack a large one") are robust. This allowed them to train the precondition model in an abstract 2D simulation and apply it successfully to a real-world 3D robot.
Experimental Results
The framework was evaluated in two primary domains: a simulated Peg-in-Hole environment and a real-world Lego stacking task using a Franka Emika Panda robot.
Peg-in-Hole (Simulation)
In a sequence of 20 tasks involving block insertion under uncertainty, the ADL framework was compared against myopic baselines, such as Confidence-Based Autonomy (which only requests help when uncertain) and Act-Learn Myopic (which ignores future task benefits). ADL consistently achieved the lowest total cost, demonstrating the value of long-term planning in skill acquisition.
Lego Stacking (Real-World)
Using a set of 15 unique Lego-style parts, the robot had to determine the most efficient way to complete a stacking sequence. The ADL planner successfully balanced its options: it recognized when its skill library was insufficient (choosing to learn or delegate) and transitioned to autonomous action once it had acquired a versatile enough skill set. The MIP formulation proved highly efficient, solving these complex sequences nearly instantaneously.
Conclusion and Future Work
The ADL framework represents a shift toward viewing robot learning as a capital investment. By integrating task allocation with learning through a MIP-based planner and a precondition prediction model, human-robot teams can achieve higher efficiency than through myopic or static strategies.
Future research directions include relaxing the assumption of fixed task sequences to allow the robot to optimize the order of tasks for better learning. Additionally, moving beyond one-to-one skill transfer toward generalized skills trained on multiple demonstrations could further enhance the framework's utility in complex, real-world environments.
Summary updated