Teach and Grow: An Agent-Centered Architecture for General Robot Learning

Chang Nie, Zhe Liu, Hesheng WangarXiv 2026 · arXiv

TL;DR

Teach-and-Grow Learning (TGL) is an agent-centered robotics architecture that eliminates the costly "retraining tax" of monolithic end-to-end models by abstracting few-shot demonstrations into reusable, closed-loop "Skill Blocks." Managed by a high-level AI agent, TGL dynamically composes task routes, adapts to real-time physical feedback, and stores experiences in a persistent library—allowing robots to continuously grow their capabilities without altering core model weights.

What Problem Does This Paper Address?

Current general-purpose robotics is dominated by end-to-end Vision-Language-Action (VLA) and world-action models. While elegant, these models are highly brittle when encountering out-of-distribution (OOD) scenarios, such as an unfamiliar object, a new sensor modality, or an unexpected physical contact.

When an end-to-end model fails locally, fixing it requires a global remedy: collecting new robot data, updating the entire policy, and running extensive regression testing to ensure old capabilities aren't broken. The authors term this recurring bottleneck the retraining tax. Unlike text-based AI, robot data cannot simply be scraped from the web; it must be generated through time-consuming physical operations.

Fig. 1. Two architectures facing the same OOD pick-and-place task. Left: an end-to-end VLA/WAM is brittle to the unfamiliar plush toy, returns to a retraining loop, and remains closed to incremental growth. Right: TGL learns from few-shot teaching, reuses Skill Blocks through an AI agent, adapts from physical feedback, and carries the acquired capability into later deployment.

TGL aims to bypass this tax by creating an architecture that can add and validate local competence incrementally without rewriting unrelated behaviors.

Methods

Instead of forcing neural network weights to absorb every nuance of the physical world, TGL introduces an agentic, modular framework that keeps foundation model weights fixed and focuses on expanding explicit skills and memory.

The architecture consists of four main pillars:

  1. The AI Agent Layer: A high-level orchestrator (powered by large multimodal models like GPT or Claude) that understands the task, retrieves appropriate skills, composes a working plan, and evaluates physical outcomes.
  2. Skill Blocks: Rather than replaying rigid trajectory fragments, a Skill Block is a closed-loop behavior defined as a contract. It encapsulates a goal, applicability scope, reusable strategy, grounding function (mapping observations to variables), compatible executors, success verifiers, and recovery choices.
  3. Skill Library: A persistent repository for verified, reusable Skill Blocks.
  4. Experience Memory: A structured textual database that records the context of successes, failures, and the specific repairs that resolved errors.
Fig. 2. Teach-and-Grow for one unfamiliar task. Teaching reveals shared state changes and new Skill Blocks. The AI Agent composes a route, runs one block, and checks its effect after the robot acts. Verified capabilities enter the Skill Library; outcomes, failures, and repairs enter Experience Memory. The complete Grow state returns to composition on the next task.

When given a new task, the robot analyzes a few successful human demonstrations to extract semantic subgoals and synthesize a shared strategy. At runtime, the agent instantiates these blocks based on the current scene geometry, executes one stage, inspects the physical consequence, and dynamically updates its route if a mismatch occurs.

Fig. 3. A Skill Block is a reusable contract for one meaningful change. It carries a goal, scope, strategy, grounding function, executor set, success test, and recovery. At runtime it observes the scene, binds current objects and geometry, chooses an executor, acts, and verifies the result.

Datasets

The authors evaluated TGL using the LIBERO benchmarking suite, a standard dataset designed for lifelong robot learning and knowledge transfer. Specifically, they utilized:

  • LIBERO-Object: For evaluating object-centric manipulation tasks.
  • LIBERO-GOAL: For analyzing feedback-driven execution and replanning traces.

Key Insights and Ideas

  • Asymmetric Architecture (Reasoning vs. Control): TGL splits the cognitive load. The language-level agent handles high-level routing, tool selection, and verification, while robot-native executors (like motion planners or low-level controllers) handle metric motion and contact. This prevents a language model from acting as a direct motor authority while still allowing physical evidence to alter the plan.
  • The "Teach-and-Grow" Scaling Law: The authors propose a new scaling hypothesis. Instead of scaling model parameters or raw dataset size, they scale effective reusable experience ($X$)—interaction data that has survived validation and remains retrievable. The hypothesis predicts that future-task error and the marginal teaching demand decay as power laws of $X$.
  • Mutual Refinement (Jiao Xue Xiang Zhang): Inspired by classical Chinese philosophy, teaching and learning promote one another. Sparse teaching seeds exploration, and the robot's subsequent failures expose gaps, making the next round of human teaching highly targeted.

Key Results

  • State-of-the-Art Performance: TGL achieves state-of-the-art success rates on the standard LIBERO benchmark.
  • Decomposition and Verification: In a visual study of 10 demonstrations across condiment-transfer tasks, TGL achieved a 1.000 ordered role/type accuracy and successfully confirmed 20/20 acquisition and release effects.
  • Local Growth Without Retraining: In a fixed-executor pilot study, a robot operating with a base library of 6 blocks scored 0/6 on evaluation tasks. After adding just 2 learned blocks to the library—with zero changes to model weights, runtime, or evaluation budgets—the success rate jumped to 4/6.

Limitations and Future Work

The primary trade-off in TGL is time and latency. Because the system relies on an agent loop that repeatedly observes, reasons, and calls tools sequentially, it is slower than feed-forward policy inference.

To address this, the authors propose a slow-teacher/fast-student extension as future work. In this paradigm, the slow agentic route explores frontiers and generates verified, structured interaction data. This data is then distilled into a fast student policy (like a VLA or diffusion policy) for low-latency, routine deployment, returning control to the agent only when uncertainty arises.

Summary updated

Abstract

End-to-end vision-language-action (VLA) and world-action models offer an elegant route to general-purpose robotics, but their reliability is bounded by validated physical coverage. When an unfamiliar object, sensor, embodiment, or contact falls outside that coverage and no validated fallback exists, correcting the failure requires new robot data, a policy update, and regression testing. This recurring burden is the retraining tax. Unlike text, embodied data must often be created by operating machines. We present Teach-and-Grow Learning (TGL), an agent-centered architecture for general robot learning. In its general form, a multimodal agent turns a few successful demonstrations into reusable Skill Blocks: closed-loop behaviors for meaningful subgoals. In a new scene, the agent grounds and composes these blocks, selects learned or geometric tools, observes the physical outcome, and revises the route when execution departs from intent. A Skill Library stores executable behavior, while structured Experience Memory carries forward success, failure, and repair. New tasks are acquired without task-specific policy retraining. Our LIBERO evaluation attains state-of-the-art performance; controlled studies expose skill induction, persistent reuse, and agent-directed adaptation. Finally, we propose the Teach-and-Grow scaling-law hypothesis: if X denotes effective reusable experience, future-task error and teaching demand should approach irreducible floors as power laws in X. The architecture therefore treats deployment as a period of continued learning, in which one task can make the next easier.
BibTeX
@misc{nie2026teach,
  author = {Chang Nie and Zhe Liu and Hesheng Wang},
  title = {{Teach and Grow: An Agent-Centered Architecture for General Robot Learning}},
  year = {2026},
  eprint = {2608.17209},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2608.17209},
}
Figure