Missing Bridges: Composition-Aware Active Imitation Learning

Maxwell J. Jacobson, Ahmed H Qureshi, Yexiang XuearXiv 2026 · arXiv

TL;DR

Traditional active imitation learning reduces expert effort by asking for demonstrations where the robot's policy is most uncertain. This paper introduces Adaptive Agents via Latent Topologies (AALT), which instead focuses on composability by building a latent map of known behaviors and requesting short "bridge" demonstrations that maximize overall start-to-goal connectivity. In a simulated robotic retrieval task, AALT achieves 100% success across all tasks using just 3 short expert queries, drastically outperforming traditional policy-centric methods.

What Problem Does This Paper Address?

In multi-task robotic domains, the number of possible start-to-goal configurations can grow combinatorially. However, many of these tasks share underlying, reusable behaviors—such as moving an obstacle out of the way before retrieving an object.

Standard active imitation learning (IL) methods are typically policy-centric: they ask the expert for full-task demonstrations based on where the learner's policy has the highest uncertainty about what action to take next. The problem? These methods don't explicitly value how well a new demonstration can be composed with existing knowledge. As a result, an expert might be asked to provide multiple long, redundant demonstrations that substantially overlap, wasting expensive human effort.

Figure 1: AALT requests demonstrations for composability. An agent begins with learned behaviors (black arrows) covering only some tasks between start states ( S ) and goal states ( A , B , C ), and may request additional expert demonstrations (orange arrows). (Left) Standard policy-centric active IL selects requests for their expected information about the expert policy. Because it does not explicitly value composability, it may request separate full-task demonstrations from S to A , B , and C , each of which may help the agent understand only one task. (Center) AALT instead organizes demonstrated behaviors into a topology connected through reusable latent hub states (blue), then requests short bridge demonstrations that maximize expected start–goal connectivity. Here, known behaviors reach h2 , while other known behaviors lead from h1 to A , B , and C . Requesting the bridge h2 →h1 creates routes to all three goals with one short demonstration. During execution, the topology specifies a route through these hubs, while a learned policy generates actions from the current observations for each successive hub transition. (Right) In the simulated retrieval task, the bridge moves the black canister to transform h2 into h1 , from which the policy can pursue all three goals. Video of the simulation is provided in the supplementary material.

Methods

AALT shifts the active learning objective from policy uncertainty to reachability information gain. The method operates in three main phases:

  1. Topology Construction (Pre-training): AALT embeds demonstrated states into a compact latent space shaped by action effects using forward and inverse dynamics models. It identifies hub states—points in the latent space where trajectories naturally converge or diverge (e.g., a specific shelf configuration after a common obstacle is moved). The demonstrated behaviors between these hubs form the directed edges of a latent topology.
  2. Composition-Aware Acquisition: Instead of asking for a full task, AALT evaluates missing edges (potential "bridges") between existing hubs. It greedily requests the single bridge demonstration that maximizes the expected increase in start-to-goal connectivity across the entire target task distribution.
  3. Inference & Execution: At test time, AALT uses Dijkstra’s algorithm to find the most reliable path through the latent topology from the start state to the goal. A goal-conditioned diffusion policy then executes the path one edge at a time, navigating from hub to hub.
Figure 2: The AALT method. (Left) latent states are clustered, and may become hub states if they converge or diverge between different demos. (Center) hubs are linked by demos to form a topology of composable behaviors. New demonstration requests are selected to optimize start–goal connectivity. Here, h3 →h1 would connect the right-side starts to the left side goals. (Right) during inference, we plan from the start state to a goal hub. A diffusion policy generates actions to reach the goal, conditioned on (hi, hj) from each edge.

Datasets

The method was evaluated on a newly designed Toolshelf domain:

  • The Setup: A simulated UR5e robot arm tasked with retrieving three ordered part canisters from a shelf containing movable access guards (blockers).
  • Task Space: 72 total tasks created by crossing 6 distinct starting shelf layouts with 12 unique multi-canister work orders.
  • Initial Data: 24 successful demonstrations (196 transitions) where the shelf was already perfectly staged for specific work orders. Crucially, 48 cross-family combinations (e.g., a specific layout paired with an unseen work order family) were completely omitted to test the agent's ability to generalize via active learning.

Key Insights and Ideas

  • The Power of Bridges: A short, simple action (like moving a single blocking canister to connect two parts of a shelf layout) can instantly unlock dozens of tasks simultaneously if it bridges two previously disconnected sub-graphs of behavioral knowledge.
  • Theoretical Grounding: The authors formally prove that reachability information gain factorizes into AALT’s connectivity gain and the uncertainty of bridge acquisition. This provides a rigorous mathematical justification for prioritizing connectivity over policy uncertainty.
  • Action-Shaped Latent Spaces: Visual similarity alone is deceptive for planning. By training the latent space using inverse dynamics, AALT ensures that states are clustered based on what the robot can do from them, rather than just what they look like.

Key Results

AALT demonstrated massive efficiency gains over state-of-the-art active learning baselines:

  • 100% Success in 3 Queries: AALT consistently achieved perfect success (72/72 tasks) using just 3 short bridge demonstrations totaling only 5 state transitions beyond the initial dataset.
  • Baseline Comparison: The strongest policy-centric baseline, Active Multi-task Fine-tuning (AMF-Full), only reached 46.9% success after 20 full-task demonstrations (171.6 transitions), largely because it wasted queries repeating variations of individual tasks it already partially understood.
Task Success vs Expert-Call Budget

Limitations and Future Work

  • Reliance on Initial Structure: AALT cannot currently propose entirely new internal hub states; it can only connect hubs that are discovered within the initial dataset or explicitly provided via start/goal states.
  • Greedy Selection: The current acquisition rule is one-step greedy. It can miss complementary bridge sets—pairs of bridges that unlock massive connectivity together but offer little value individually. The authors suggest multi-step lookahead or beam search as future solutions.
  • Skill Refinement: AALT assumes that if a route exists in the topology, it can be executed. If an existing edge behavior is poorly learned and unreliable, AALT won't request a demonstration to fix it. A promising future direction is a hybrid objective combining AALT's connectivity gain with policy-centric uncertainty refinement.

Summary updated

Abstract

Active imitation learning reduces expert effort by allowing a learner to request the demonstrations it needs. Existing methods typically select these requests for their expected information gain about the expert policy. In structured multi-task domains, however, the number of start-goal tasks may grow combinatorially despite their solutions sharing reusable behavior. This makes composable behaviors especially valuable, since a single demonstration may help solve many tasks at once. Prior methods do not explicitly account for this value when selecting which demonstration to request. We introduce Adaptive Agents via Latent Topologies (AALT), which requests demonstrations that maximize expected gains in start-goal connectivity. We further show that this objective is formally tied to information gain about task reachability. AALT organizes existing demonstrations into a topology of latent hub states connected by learned behaviors, identifies high-value bridge demonstrations that are likely to enable many tasks at once, and grounds each to an expert query. At inference, it plans through the resulting topology and conditions a diffusion policy on each successive hub transition. In a simulated UR5e robot ordered-retrieval domain with 72 tasks, AALT improved from 42/72 to 72/72 (100%) successful tasks consistently using only 3 demonstrations totaling 5 transitions beyond the initial dataset. After 20 demonstrations, the strongest baseline averaged 88.6% success using 98 transitions.
BibTeX
@misc{jacobson2026missing,
  author = {Maxwell J. Jacobson and Ahmed H Qureshi and Yexiang Xue},
  title = {{Missing Bridges: Composition-Aware Active Imitation Learning}},
  year = {2026},
  eprint = {2609.18004},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.18004},
}
Figure