Certified Long-Horizon Code Agent Evolution via Validation-Gated Skill Optimization

Yifan Wang, Hao Cheng, Xiaomin Li, Yuexing Hao, Hemanth Neelgund Ramesh, Dongwon Jung, Hao Tang, Keru Wang, Chenliang Zhou, Qianhui Wu, Wenlin Yao, Ananth Grama, Andrzej Banburski-Fahey, Baolin Peng, Jaron Lanier, Jianfeng GaoarXiv 2026 · arXiv

TL;DR

This paper introduces VALVE, a framework that enables LLM-based code agents to "self-evolve" over long deployments without updating model weights. By implementing a rigorous validation gate that only accepts new skills if they improve performance on a held-out task panel, the authors prevent "agent collapse" and achieve a 75% reduction in performance drawdown while maintaining an 11x more compact skill bank.

What Problem Does This Paper Address?

Large Language Model (LLM) agents are increasingly used for complex, multi-step tasks like repository-level software engineering. Ideally, these agents should learn from their experiences, accumulating a "skill bank" to solve future problems more efficiently. This is known as in-context self-evolution.

However, existing methods for self-evolution often fail over long horizons (e.g., 1,000+ sequential tasks). As the agent accumulates more "skills," it often picks up noisy, overly specific, or flat-out wrong information. This leads to agent collapse: a phenomenon where the agent’s performance peaks early and then degrades significantly (drawdown) as the memory bank becomes cluttered and misleading. The authors aim to provide both a practical framework and theoretical guarantees to ensure that an agent's learning is stable, cumulative, and "certified" to improve over time.

Methods

The authors formalize the evolution process into a three-step loop called Extract-Validate-Retrieve (EVR):

  1. Extract (Contrastive Reflection): When an agent attempts a task and gets mixed results (some attempts succeed, others fail), it uses an LLM to contrast the successful and failed trajectories. It extracts two types of knowledge:
  2. Principles: High-level, domain-agnostic reasoning strategies (e.g., "check for reordered statements").
  3. Skills: Concrete, executable procedures (e.g., a specific regex fix for a library).
  4. Validate (The Gate): This is the core innovation. Before a new skill or principle is added to the permanent memory bank, it must pass a "validation gate." The agent runs a small, fixed panel of held-out tasks using the proposed update. The update is only committed if the performance improves by a specific margin ($\rho$).
  5. Retrieve: For every new task, an embedding-based retriever selects the top-10 most relevant principles and skills from the bank to inject into the agent's prompt.
Figure 2: Overview of VALVE.

Beyond the framework, the paper provides a theoretical foundation, establishing finite convergence (the bank won't change forever) and deriving the necessary sizes for validation and evaluation sets to guarantee that measured gains are statistically significant.

Datasets

The study uses SWE-smith, a large-scale benchmark for repository-level software engineering. They focus on four diverse Python repositories:

  • Pygments: Syntax highlighting.
  • sqlfluff: SQL parsing and linting.
  • pydicom: Medical image processing.
  • deepdiff: Recursive object comparison.

Each repository is split into a 100-task evaluation set, a 50-task validation panel, and a stream of approximately 1,000 tasks for the evolution process.

Key Insights and Ideas

  • Less is More: The validated agent produced an 11x smaller skill bank than the ungated version. By being "picky," the agent avoids the "context clutter" that leads to performance collapse.
  • Stability via Gating: Without validation, agents often suffer from "transient gains"—they look like they are learning, but they are actually overfitting to recent tasks at the expense of general capability.
  • Contrastive Learning is Crucial: Learning from why a specific attempt failed compared to why another succeeded provides much richer signal than just looking at successful outcomes.
  • Theoretical Certification: The authors prove that you can calculate exactly how many tasks you need in your validation set to ensure that your agent's "drawdown" (performance loss) stays below a certain threshold with high probability.

Key Results

The authors tested VALVE across three frontier models: GPT-5.5, Claude-4.6, and MiniMax-M2.7.

  • Performance Gains: VALVE achieved an average final gain of 14.9 points and a peak gain of 16.5 points across all models and repositories.
  • Drawdown Reduction: The validation gate reduced average drawdown by 75%. As seen in Figure 1, while an ungated agent's performance might crash after a few hundred steps, the VALVE-gated agent remains stable.
  • Efficiency: The resulting skill banks were significantly more compact, making them easier to manage and cheaper to use in terms of prompt tokens.
Figure 1: Validation stabilizes long-horizon self-evolution. (a) Avg@3 resolved rate over approximately 1,000 SWE-smith task arrivals. Both runs improve early, but the ungated run falls from a 91.7% peak to 80.0%, while the validated run peaks at 97.0% and ends at 92.3%. (b) Across all twelve model–repository runs, validation reduces average drawdown by 75%.

Limitations and Future Work

  • Computational Cost: Running a 50-task validation panel for every potential update is expensive in terms of both time and API tokens. The authors note that validated evolution takes roughly 2.4x longer than ungated evolution.
  • Static Retriever: The current framework keeps the retrieval mechanism fixed. Future work could explore "evolving the retriever" alongside the skill bank.
  • Distribution Shift: The theoretical guarantees assume tasks are drawn from a stationary distribution. If the nature of the software engineering tasks changes radically mid-stream, the fixed validation panel might become less effective.

Summary updated

Abstract

Long horizon agent self-evolution without model weight updates is essential for enabling deployed agents to accumulate reusable skills and improve over time. Prior self-evolution work has focused primarily on short-horizon tasks, while repository-level software engineering remains unexplored despite being an ideal testbed for long-horizon adaptation. In this setting, agents are required to solve streams of sequential tasks, navigate complex dependencies with evolving repositories and persistently store and reuse experience. Text-based skill optimization offers an efficient, non-parametric approach for such adaptation. However, existing methods often suffer from unstable updates, performance drawdown, and agent collapse over extended deployments. In this paper, we formalize the concept of in-context self-evolution and introduce VALVE, a validated-gated framework for long-horizon skill optimization. We establish finite convergence, provide theoretical guarantees for future-task gain and drawdown, and derive the validation and evaluation holdout sizes required for a prescribed tolerance, with leading-order scaling Empirically, our pipeline, VALVE achieves stable self-improvement over evolution horizon spanning more than 1,000 SWE tasks, with average final and peak gains of $14.9$ and $16.5$ points across three frontier models (GPT-5.5, Claude-4.6 and MiniMax-M2.7). The validation gate reduces average drawdown by 75% and produces an 11x more compact skill bank than ungated evolution. We further present extensive ablations identifying the design choices most critical to long-horizon skill evolution.
BibTeX
@misc{wang2026certified,
  author = {Yifan Wang and Hao Cheng and Xiaomin Li and Yuexing Hao and Hemanth Neelgund Ramesh and Dongwon Jung and Hao Tang and Keru Wang and Chenliang Zhou and Qianhui Wu and Wenlin Yao and Ananth Grama and Andrzej Banburski-Fahey and Baolin Peng and Jaron Lanier and Jianfeng Gao},
  title = {{Certified Long-Horizon Code Agent Evolution via Validation-Gated Skill Optimization}},
  year = {2026},
  eprint = {2609.32990},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.32990},
}
Figure