Zero-Shot Composition of Modular Skill Abstractions

This direction is split into other directions: modular skill transfer. Its last version stays readable and citable here.

Version 1 of 1370 papers427 conceptsfrozen 2026-10-04

When a human is asked to "make breakfast," they don't think about the precise torque required for every finger joint. Instead, they reason in high-level, reusable subtasks: crack egg, toast bread, pour juice. This hierarchical structure is the foundation of our ability to solve complex, long-horizon problems.

In contrast, traditional "flat" reinforcement learning policies often struggle to generalize. If a robot learns to "pick up a red block and put it in a blue bowl," it might completely fail if asked to put the block in a green bowl, or to pick up a spoon instead. The research direction emerging across these papers moves away from monolithic policies toward Modular Skill Abstractions: a paradigm where agents learn a library of discrete, atomic behaviors that can be recombined at runtime to solve entirely new tasks.

The Bottleneck of Continuity

The primary challenge in scaling embodied AI is the "curse of dimensionality" in long-horizon tasks. If an agent treats every unique sequence of actions as a new task to be learned from scratch, the data requirements become astronomical.

To solve this, researchers like Divyansh Garg and Stefano Ermon in LISA propose using vector quantization to force an agent's internal representations into discrete "skills." By discretizing the latent space, the model is forced to ignore the noise of specific trajectories and focus on the "atoms" of behavior. This modularity allows for compositional generalization: the ability to execute a multi-step instruction like "open the drawer, then pick up the knife" even if that specific sequence was never seen during training.

Decoupling the What from the How

A recurring theme in this field is the separation of high-level intent from low-level execution. Haozhe Shan and Lea Duncker argue that by separating the inference of what computation to perform from how to perform it, models can avoid the "catastrophic forgetting" that plagues multi-task learning.

This decoupling takes several forms:

  1. Symbolic Operators: In the Ada framework, Jiayuan Mao and Pratyusha Sharma use Large Language Models (LLMs) to propose symbolic operators—high-level action descriptions—that are then verified through environment interaction.
  2. Task Graphs: De-An Huang and Danfei Xu introduced Neural Task Graphs, which represent tasks as nodes (actions) and edges (state transitions). This allows a robot to watch a single video demonstration and understand the underlying "logic" of the task, enabling it to recover from errors or handle different object permutations.
  3. Temporal Logic: For safety-critical tasks, Wenjie Qiu and He Zhu use Linear Temporal Logic (LTL) to chain low-level navigation skills. This ensures that an agent can follow complex, "infinite-horizon" instructions—like "Visit A, then B, while avoiding C"—with formal guarantees.

Learning from Messy Data

One of the most powerful aspects of modular abstractions is their ability to extract structure from unstructured data. Lili Chen and Deepak Pathak demonstrate in PlayFusion that agents can learn a "vocabulary" of motion primitives from uncurated "play" data.

By treating robotic behaviors like words in a sentence, models like QueST can perform few-shot adaptation. If a robot already knows the "tokens" for reach, grasp, and lift, learning to "pick up a new fruit" becomes a matter of finding the right sequence of existing tokens rather than learning a new motor policy from scratch.

Why It Works: The Power of Recombination

The intuition behind this research is mathematical: if you have 100 primitive skills, the number of 5-step sequences you can form is $100^5$. A monolithic model would need to see a representative sample of those 10 billion combinations to master them. However, a modular system only needs to master the 100 primitives and the "grammar" of how they connect.

This is why Kevin Leahy and colleagues can achieve zero-shot task composition using Boolean logic. By composing pre-trained skills with operators like AND, OR, and NOT, agents can navigate complex environments with multiple constraints at deployment time without any additional training.

Ultimately, this direction suggests that the path to general-purpose robots lies not in bigger end-to-end networks, but in more flexible, modular "alphabets" of behavior that can be composed, verified, and re-planned on the fly.

Papers in this version

Showing 60 of 370 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_zero_shot_composition_of_modular_skill_a_8981cf5d,
  title = {Zero-Shot Composition of Modular Skill Abstractions},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/zero-shot-composition-of-modular-skill-abstractions/rdr_8981cf5dfd5241f58c94a12e789a1119},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_zero_shot_composition_of_modular_skill_a_8981cf5d_v1,
  title = {Zero-Shot Composition of Modular Skill Abstractions},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/zero-shot-composition-of-modular-skill-abstractions/rdr_8981cf5dfd5241f58c94a12e789a1119/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}