Compounding Physical RSI: Turning Execution Failure into Inherited Capability

Yixuan Yang, Qiang Chen, Run Shao, Shuwei Dong, Wenhao Li, Yunpeng Wang, Wenhan Fang, Haomiao Wu, Xingyu Wang, Lichen Man, Rong Zhang, Xuezhi Cui, Yuan Liu, Bingyan Lu, Jiawei Ma, Xiaoyang Ma, Zhemin Sun, Zhuorui Yao, Yutong Dong, Yuxuan Zhao, Yichong Mu, Jiabao Jiang, Wenxuan Deng, Junling Zhou, Yi Chen, Deyu Zhang, Yiming Yan, Yige Yuan, Shan You, Fei Wang, Haifeng Li, Hongyan Xu, Yichao Cao, Xiu SuarXiv 2026 · arXiv

TL;DR

This paper introduces Compounding Physical Recursive Self-Improvement (RSI), a framework that enables robot policies to systematically learn from their own execution failures. By diagnosing specific stage-level bottlenecks and using weight-space consolidation to prevent skill regression, the authors produced CSU-AI-07, a policy that currently dominates the RoboDojo-Sim benchmark with a 39% relative improvement in success rate over previous state-of-the-art models.

What Problem Does This Paper Address?

Training generalist robot policies often relies on static offline datasets. However, physical manipulation is inherently sequential: a robot cannot learn how to fail at "placing" an object if it cannot first "grasp" it. The authors identify a capability frontier—the earliest stage in a task where a policy reliably fails. Once one bottleneck is fixed, the next hidden failure mode is exposed.

The challenge is that standard improvement loops often fail to "compound" for three reasons:

  1. Opaque Credit Assignment: A simple "failure" flag doesn't tell the robot which stage (approach, grasp, transport) actually broke.
  2. Lack of Corrective Supervision: Failed rollouts show what went wrong, but not the "counterfactual" action that would have succeeded.
  3. Catastrophic Forgetting: In a multi-task setting, updating weights to fix a specific failure in one task often degrades performance in others.

Methods

The authors propose a three-stage "Diagnosis–Remedy–Retention" framework to ensure that improvements in one generation are inherited by the next.

The three-stage architecture of Compounding Physical RSI, showing the flow from diagnosis to candidate promotion.
  • Diagnosis: Instead of binary success/failure, an automated operator traces rollouts to pinpoint the exact task stage where progression halts. This generates a structured improvement objective ($g_k$) targeting specific issues like grasp stability or contact persistence.
  • Remedy: The system curates a targeted experience subset from three sources: task exemplars (expert demos), geometric exemplars (spatial structure), and on-policy execution evidence. It then uses behavior-regularized offline reinforcement learning to refine the policy against the improvement objective without requiring expensive new online rollouts during the optimization step.
  • Retention: To prevent regression, the framework doesn't just adopt the new weights. It constructs a trajectory of candidate policies in weight-space between the old "reference" policy and the new "refined" policy (using linear interpolation). These candidates are evaluated across the entire benchmark, and only the best-performing one is promoted to the next generation.

Datasets

The primary environment for training and evaluation is RoboDojo-Sim, a demanding benchmark consisting of 42 tasks that test generalization, memory, long-horizon execution, and precision. The framework also leverages an experience base built from:

  • Task Exemplars: Instruction-conditioned multi-stage sequences.
  • Geometric Exemplars: Target-conditioned spatial data.
  • On-policy Execution Evidence: Data collected by the robot’s current policy, including near-failure recovery behaviors.

Key Insights and Ideas

  • Success as a Product: The authors argue that overall task success is the product of stage-level pass rates. Therefore, even small gains in an early stage (like stabilizing a grasp) multiply the number of rollouts that reach later stages, providing richer data for future generations.
  • The Moving Frontier: Improvement is most consequential at the current capability frontier. By focusing updates there, the robot systematically "unlocks" the ability to see and fix downstream failures.
  • Weight-Space Consolidation: By searching the linear path between old and new parameters, the framework finds a "Pareto frontier" that balances new capability acquisition with the preservation of previously mastered skills.
Two case studies showing how Diagnosis localizes failures to specific stages (e.g., interaction vs. transport) to guide the Remedy stage.

Key Results

The resulting policy, CSU-AI-07, achieved significant margins on the RoboDojo-Sim leaderboard:

  • Overall Score: 26.15 (a 29% relative gain over the next-best entry, GalaxeaVLA).
  • Success Rate: 20.70% (a 39% relative gain over the next-best entry).
  • Stability: The results were consistent across three independent evaluation seeds, confirming that the evaluation-guided promotion process yields stable generalization rather than narrow overfitting.

Limitations and Future Work

  • Improvement Operators: Currently, the improvement operators (the algorithms for diagnosis and remedy) are fixed. The authors suggest that a higher bar for RSI would be to evolve the improvement procedure itself.
  • Sim-to-Real: While the framework is tested in simulation, the authors acknowledge that moving to physical hardware—where each trial is significantly more expensive—will be the ultimate test of the framework's efficiency.
  • Ablation Studies: The current report evaluates the integrated system; the authors note that future work will involve fine-grained ablations to isolate the specific contributions of diagnosis vs. weight consolidation.

Summary updated

Abstract

Generalist robot policies are trained on static offline datasets, yet physical manipulation unfolds sequentially across stages. Because a weakness in a downstream stage remains unobservable until the policy passes earlier ones, each resolved bottleneck exposes the next hidden failure mode. This staged structure calls for Recursive Self-Improvement (RSI), in which a policy iteratively executes, diagnoses failures, and updates itself. Moreover, because task completion behaves like a product of stage-level pass rates, recursive gains can multiply across generations. In physical environments, three bottlenecks obstruct compounding: terminal execution feedback rarely localizes the stage that caused a failure, failed trajectories do not supply the corrective action, and updating shared parameters can degrade previously mastered skills. We propose Compounding Physical RSI, a self-evolving policy framework that turns execution failures into heritable capability. First, automated capability-gap assessment traces failed rollouts to the task stage where progression breaks down and formulates a structured improvement objective. Second, conditioned on this objective, our remedy stage curates validated, stage-relevant experience and refines the policy via behavior-regularized offline reinforcement learning, without requiring costly online rollouts at each optimization step. Third, to safeguard existing skills, our retention stage constructs weight-space candidates between the inherited reference policy and the refined policy, evaluates them across the task suite, and promotes the best candidate as the reference state for the next generation. Evaluated across all 42 tasks of the RoboDojo-Sim benchmark, CSU-AI-07 ranks first on the leaderboard with a three-seed mean Overall Score of 26.15 and Success Rate of 20.70%, outperforming the next-best entry by 5.92 points and 5.82 percentage points.
BibTeX
@misc{yang2026compounding,
  author = {Yixuan Yang and Qiang Chen and Run Shao and Shuwei Dong and Wenhao Li and Yunpeng Wang and Wenhan Fang and Haomiao Wu and Xingyu Wang and Lichen Man and Rong Zhang and Xuezhi Cui and Yuan Liu and Bingyan Lu and Jiawei Ma and Xiaoyang Ma and Zhemin Sun and Zhuorui Yao and Yutong Dong and Yuxuan Zhao and Yichong Mu and Jiabao Jiang and Wenxuan Deng and Junling Zhou and Yi Chen and Deyu Zhang and Yiming Yan and Yige Yuan and Shan You and Fei Wang and Haifeng Li and Hongyan Xu and Yichao Cao and Xiu Su},
  title = {{Compounding Physical RSI: Turning Execution Failure into Inherited Capability}},
  year = {2026},
  eprint = {2611.06654},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.06654},
}
Figure