RoboCoEvo: Expanding Robot Capabilities through Agent-Policy Co-Evolution

Ran Qi, Yixin Xiao, Dawei Wang, Zhiyuan Li, Jinbang Huang, Xuan Zhao, Yintao Ma, Yangzheng Wu, Tongtong Cao, Yingxue ZhangarXiv 2026 · arXiv

TL;DR

RoboCoEvo is a framework that creates a virtuous cycle between a robotic policy and an MLLM-based agent. By having the agent identify recurring failure modes and write executable recovery code, the system generates targeted corrective data that the policy "internalizes" through fine-tuning, leading to sustained performance gains across successive rounds of co-evolution.

What Problem Does This Paper Address?

Current robotic systems often combine a low-level policy (like a Vision-Language-Action model) with a high-level agent (like an MLLM) that acts as a supervisor. However, these two components are usually developed in isolation.

This decoupling leads to two main issues:

  1. Dependency: The policy never learns from the agent’s interventions, remaining dependent on the agent to fix the same mistakes repeatedly.
  2. Stagnation: The agent’s knowledge remains static. As the policy improves and its failure patterns shift, a static agent continues to target weaknesses that may no longer exist while missing new, emerging bottlenecks.

RoboCoEvo addresses this by treating policy learning and agent adaptation as a coupled, iterative process.

Methods

The RoboCoEvo framework operates in two main phases that repeat in "rounds":

  1. Policy-Guided Agent Evolution:
    • The system collects rollouts from the current policy and identifies failures.
  2. An MLLM agent groups these failures into Failure Modes (FMs) based on recurring patterns (e.g., "slipping while grasping a jar").
  3. The agent then writes Detectors (to find these states in future rollouts) and Recovery Programs (code-as-policy scripts that use motion planners to complete the failed subtask).
  4. Agent-Guided Policy Capability Expansion:
    • Failure-Conditioned Augmentation: The system takes a failed state and perturbs it in the simulator (changing positions slightly) while ensuring the failure mode remains the same.
  5. The recovery programs are run on these states to generate a large volume of "corrective demonstrations."
    • The policy is fine-tuned on a mix of original successful demonstrations and this new corrective data, allowing it to "internalize" the recovery behavior.

In the next round, the updated policy is deployed, and the agent revises its programs to address the new distribution of failures.

Datasets

The authors evaluated RoboCoEvo on three diverse simulation benchmarks:

  • BEHAVIOR-1K: Challenging, long-horizon bimanual mobile manipulation tasks in household settings.
  • RoboTwin: Bimanual tabletop manipulation tasks requiring high coordination.
  • RoboCasa: Realistic kitchen environments using a single-arm mobile manipulator.

Key Insights and Ideas

  • Targeted Data is King: The researchers found that training on data specifically targeting the policy's failures is significantly more effective than simply adding more successful demonstrations. For example, "success-only" data required nearly double the GPU hours and significant human effort to achieve lower performance than the automated RoboCoEvo approach.
  • Consistency via Shared Programs: Instead of generating a unique recovery for every single failure, grouping failures into "modes" and using a shared recovery program produces more consistent data. This consistency makes it much easier for the neural network policy to learn the underlying pattern.
  • The "Moving Bottleneck" Phenomenon: In long-horizon tasks (like "canning meat"), the policy's progress is often hidden. In Round 1, the policy might fail to even get a jar out of a cabinet. Once it learns that, Round 2 reveals new failures in opening the lid. Co-evolution is necessary to chase these shifting bottlenecks down the task chain.
  • Efficiency through Augmentation: By perturbing failed states in the simulator, the system can generate 3.6x more training data without needing to run more expensive, full-task rollouts.

Key Results

RoboCoEvo demonstrated sustained improvements across all benchmarks over two rounds of evolution:

  • RoboTwin: Success rates jumped from 79% to 93%.
  • RoboCasa: Success rates increased from 79% to 91%.
  • BEHAVIOR-1K: The mean Q-score (a measure of subtask completion) rose from 0.40 to 0.56.
  • Failure Reduction: The fraction of episodes failing due to "Targeted Failure Modes" dropped drastically (e.g., from 0.188 to 0.058 on RoboTwin), proving the policy successfully internalized the agent's corrections.

Limitations and Future Work

  • Success Bias: Currently, the framework only learns from successful agent recoveries. It ignores the potential information contained in failed recovery attempts.
  • Lack of Reinforcement Learning: The system relies on imitation learning (DAgger-style). Integrating Reinforcement Learning could allow the policy to explore and optimize beyond the specific recovery programs written by the agent.
  • Simulation Dependency: The current failure-conditioned augmentation relies on having access to the simulator state to perturb objects and robots, which may be harder to implement in real-world settings without high-fidelity digital twins.

Summary updated

Abstract

Integrating MLLM-based agents with robotic foundation models has attracted increasing attention as a means of enhancing robotic capabilities. However, the improvement of such integrated systems remains largely decoupled across their agent and model components, with advances in one component not necessarily benefiting the other, substantially limiting the capabilities of the combined system. As both MLLMs and robotic foundation models become increasingly capable, fully realizing their combined potential calls for co-evolution, an iterative process in which improvements in agents and robotic foundation models reinforce one another. We propose RoboCoEvo, a failure-guided co-evolution framework that couples the refinement of the agent's explicit and reusable executable knowledge with parameter learning in robotic foundation models, such as vision-language-action (VLA) models. Through a code-as-policy formulation, the agent develops and refines executable failure-detection and recovery programs, generating targeted corrective experience. Post-training consolidates the resulting corrective behaviors into the robotic model's end-to-end policy, improving autonomous task execution. Changes in the updated policy's failure distribution, in turn, guide subsequent revisions to the agent's programs. Experiments on BEHAVIOR-1K, RoboCasa, and RoboTwin demonstrate sustained improvements in autonomous VLA task performance across successive co-evolution rounds.
BibTeX
@misc{qi2026robocoevo,
  author = {Ran Qi and Yixin Xiao and Dawei Wang and Zhiyuan Li and Jinbang Huang and Xuan Zhao and Yintao Ma and Yangzheng Wu and Tongtong Cao and Yingxue Zhang},
  title = {{RoboCoEvo: Expanding Robot Capabilities through Agent-Policy Co-Evolution}},
  year = {2026},
  eprint = {2611.19081},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.19081},
}
Figure