TL;DR
This survey maps out a fundamental fork in the road for robot learning: should we bake robotic competence into the frozen weights of monolithic neural networks, or build agents that write and refine their own executable code? By analyzing over 300 papers, the authors establish a clear taxonomy based on this "weights versus skills" axis and outline a "self-improvement ladder" for code-generating robots. Finally, they connect these paradigms to the newly emerging commercial "robot app store" economy, highlighting the massive gap between static code playback and true on-device adaptation.
What Problem Does This Paper Address?
The field of robot learning is growing rapidly, but it is splitting into two fundamentally different philosophies. On one side, researchers are scaling up end-to-end Vision-Language-Action (VLA) models that map observations directly to motor commands. On the other side, "code-as-policy" approaches use Large Language Models (LLMs) to generate inspectable, executable programs that orchestrate low-level control primitives.
Because the field has moved so fast, the literature has become cluttered, and terms like "skill" are heavily overloaded. Furthermore, while code-based robots have the unique ability to improve without gradient updates, there hasn't been a structured way to categorize how they improve. This paper addresses this gap by providing a rigorous taxonomy of the field, defining exactly what constitutes robotic "self-improvement," and identifying the open technical challenges preventing downloaded robot skills from safely adapting to new environments.
Methods
The authors organize 77 core systems (and catalog 225 more in an extensive appendix) into a taxonomy structured around six technique families.
The core analytical contribution is the Code-as-Policy Self-Improvement Ladder, which ranks systems into five rungs based on three operational mechanisms:
- Feedback (F): The ability to read an execution-grounded signal (like a failure detector) and revise code within a task.
- Memory (M): Storing validated code in a persistent library to reuse across different tasks.
- Search (S): Maintaining and mutating a population of candidate programs (e.g., evolutionary algorithms) rather than just following a single line of repairs.
The rungs range from simple Zero-shot synthesis (writing code once; if it fails, it fails) to the Full self-improving loop (combining Feedback, Memory, and Search into an open-ended loop). Currently, only a few very recent systems like ASPIRE, ENPIRE, and RoboClaw occupy this top rung.
Datasets
The paper reviews systems evaluated across a broad spectrum of established simulation benchmarks and real-world datasets, including:
- Open X-Embodiment: The massive multi-robot dataset used to train generalist VLA models.
- LIBERO: A benchmark specifically designed for lifelong robot learning and knowledge transfer.
- Simulation Suites: Meta-World, robosuite, BEHAVIOR-1K, RLBench, ManiSkill2, and CALVIN.
Instead of introducing a new dataset, the authors propose a new actionable evaluation protocol consisting of four concrete metrics for future benchmarks: the success-vs-interactions curve, skill-library reuse rate, cross-embodiment transfer drop, and provenance coverage.
Key Insights and Ideas
- The Five Senses of "Skill": The survey reveals that the word "skill" is used in five distinct ways in literature—ranging from a latent reinforcement learning policy to a commercial product in an app store. Crucially, only the code representation is simultaneously human-inspectable, adaptable after creation, and symbolically composable.
- The Short-Loop Limitation: Most closed-loop repair systems suffer from a "short loop"—they fix code on the fly when a robot encounters an error, but they discard that correction at the task boundary. Competence only truly compounds when code is written to a persistent cross-task memory.
- The Static-to-Adaptive Marketplace Gap: Commercial marketplaces like Unitree's UniStore have arrived, allowing users to download robot skills with one tap. However, these are currently just "static playbacks" (animations). The real frontier of physical AI requires "Layer 3" adaptation—such as adjusting a grip dynamically so a downloaded skill doesn't bruise a specific type of fruit.
Key Results
The paper provides a comprehensive Family Comparison Matrix evaluating the trade-offs of each paradigm:
- End-to-End VLAs excel at short, reactive chunks and cross-embodiment transfer via massive data scaling, but they suffer from opaque weights, silent misgrounding under distribution shifts, and high data demands.
- Code-as-Policy requires only few-shot prompts and generates readable, editable, long-horizon programs, but it can be incredibly brittle if the underlying perception stack mislabels objects in the scene.
- Reward Synthesis (e.g., Eureka) successfully uses LLMs to write reward functions optimized by reinforcement learning, but it is prone to "reward hacking" where the policy exploits simulation physics.
Limitations and Future Work
The authors explicitly note that this survey is intentionally code-centric; it treats representation learning and pure reinforcement learning as context rather than the core focus. Additionally, the "skill economy" discussion is framed around an early commercial trend rather than a mature market.
For future work, they advocate for three directions:
- Standardized Self-Improvement Benchmarks: Creating protocols that explicitly plot a robot's success rate against its autonomous interaction budget.
- Marketplace Adaptation Primitives: Shifting the paradigm so a downloaded robot skill is treated as an initial optimization starting point rather than a rigid, frozen asset.
- Bridging RL and Code Senses: Developing hybrid interfaces where executable code can wrap, call, and refine latent reinforcement learning policies, combining the best of both worlds.
Summary updated