Abstract
Robots deployed in many real-world settings need to be able to acquire new skills and solve new tasks over time. Prior works on planning with skills often make assumptions on the structure of skills and tasks, such as subgoal skills, shared skill implementations, or task-specific plan skeletons, which limit adaptation to new skills and tasks. By contrast, we propose doing task planning by jointly searching in the space of parameterized skills using high-level skill effect models learned in simulation. We use an iterative training procedure to efficiently generate relevant data to train such models. Our approach allows flexible skill parameterizations and task specifications to facilitate lifelong learning in general-purpose domains. Experiments demonstrate the ability of our planner to integrate new skills in a lifelong manner, finding new task strategies with lower costs in both train and test tasks. We additionally show that our method can transfer to the real world without further fine-tuning.
Introduction
For a robot to be truly versatile in dynamic environments, it must possess the ability to acquire new skills over its lifetime and determine how to combine them to solve novel tasks. Traditional task planning approaches often struggle with this lifelong learning requirement because they rely on rigid assumptions. These include requiring skills to reach specific, predefined subgoals, assuming shared underlying implementations, or depending on task-specific plan skeletons.
Such rigidity becomes a bottleneck when a robot learns a fundamentally new type of skill—for instance, using a tray to sweep multiple objects simultaneously rather than picking them up individually. Existing planners typically cannot incorporate such diverse skills without extensive manual re-engineering. This work addresses the challenge of building a flexible task planner that can dynamically search through an ever-growing library of diverse, parameterized skills to solve new tasks on the fly.
Methodology: Search-Based Planning with SEMs
The proposed framework combines search-based task planning with learned Graph Neural Network (GNN) models that predict the outcomes of skills.
Skill Effect Models (SEMs)
Instead of relying on computationally expensive low-level simulations for every action, the system utilizes GNN-based SEMs to predict the "macro-outcome" of a skill. The GNN treats objects in the scene as nodes in a graph. Given the current state and a set of skill parameters, the SEM predicts the final state of the objects and the associated execution cost. This macro-modeling approach drastically reduces the planning horizon, making search-based planning computationally feasible.
Iterative Data Collection and Search
To train these models efficiently, the authors employ an iterative training procedure. The planner (using Weighted A*) attempts to solve training tasks using its current SEMs. These plans are executed in simulation to collect ground-truth data, which is then used to update and refine the SEMs. This planner-guided exploration is highly sample-efficient because the planner acts as a funnel, directing learning toward the most useful regions of the state space.
Because the planner searches directly in the space of continuous states and skill-parameter tuples, adding a new skill is straightforward: it simply becomes a new branching option during the A* search. This allows for zero-shot task generalization, where newly learned skills can be applied to unseen tasks just by updating the goal condition in the search.
Experimental Evaluation
The method was evaluated in a custom block manipulation domain built in the Nvidia Isaac Gym simulator, utilizing a Franka Emika Panda arm. The environment contains blocks, a table, a tray, and a bin.
Key Results
- Computational Efficiency: Planning with learned SEMs takes approximately 1 second, a massive speedup compared to the 10 to 30 minutes required when using a ground-truth simulator for planning.
- Lifelong Skill Integration: As new skills (e.g., Tray Slide, Tray Sweep, or Bin Tilt) were added, the planner successfully discovered new, lower-cost strategies. For instance, the addition of the Tray Slide skill improved the success rate on difficult "Far Bin" tasks from 0% to 100%.
- Sim-to-Real Transfer: SEMs trained entirely in simulation successfully transferred to a physical robot replica without any fine-tuning, achieving success rates between 80% and 100% on real-world test tasks.
Conclusion and Future Work
This search-based task planning framework demonstrates that macro-models of skills can enable efficient, lifelong learning in robotic manipulation. While successful, future work could improve the system by implementing learned parameter samplers to handle massive parameter spaces more effectively. Additionally, incorporating uncertainty estimation into the SEMs could help the planner avoid unknown regions or actively target areas for model fine-tuning.
Summary updated