TL;DR
This paper introduces VALVE, a framework that enables LLM-based code agents to "self-evolve" over long deployments without updating model weights. By implementing a rigorous validation gate that only accepts new skills if they improve performance on a held-out task panel, the authors prevent "agent collapse" and achieve a 75% reduction in performance drawdown while maintaining an 11x more compact skill bank.
What Problem Does This Paper Address?
Large Language Model (LLM) agents are increasingly used for complex, multi-step tasks like repository-level software engineering. Ideally, these agents should learn from their experiences, accumulating a "skill bank" to solve future problems more efficiently. This is known as in-context self-evolution.
However, existing methods for self-evolution often fail over long horizons (e.g., 1,000+ sequential tasks). As the agent accumulates more "skills," it often picks up noisy, overly specific, or flat-out wrong information. This leads to agent collapse: a phenomenon where the agent’s performance peaks early and then degrades significantly (drawdown) as the memory bank becomes cluttered and misleading. The authors aim to provide both a practical framework and theoretical guarantees to ensure that an agent's learning is stable, cumulative, and "certified" to improve over time.
Methods
The authors formalize the evolution process into a three-step loop called Extract-Validate-Retrieve (EVR):
- Extract (Contrastive Reflection): When an agent attempts a task and gets mixed results (some attempts succeed, others fail), it uses an LLM to contrast the successful and failed trajectories. It extracts two types of knowledge:
- Principles: High-level, domain-agnostic reasoning strategies (e.g., "check for reordered statements").
- Skills: Concrete, executable procedures (e.g., a specific regex fix for a library).
- Validate (The Gate): This is the core innovation. Before a new skill or principle is added to the permanent memory bank, it must pass a "validation gate." The agent runs a small, fixed panel of held-out tasks using the proposed update. The update is only committed if the performance improves by a specific margin ($\rho$).
- Retrieve: For every new task, an embedding-based retriever selects the top-10 most relevant principles and skills from the bank to inject into the agent's prompt.
Beyond the framework, the paper provides a theoretical foundation, establishing finite convergence (the bank won't change forever) and deriving the necessary sizes for validation and evaluation sets to guarantee that measured gains are statistically significant.
Datasets
The study uses SWE-smith, a large-scale benchmark for repository-level software engineering. They focus on four diverse Python repositories:
- Pygments: Syntax highlighting.
- sqlfluff: SQL parsing and linting.
- pydicom: Medical image processing.
- deepdiff: Recursive object comparison.
Each repository is split into a 100-task evaluation set, a 50-task validation panel, and a stream of approximately 1,000 tasks for the evolution process.
Key Insights and Ideas
- Less is More: The validated agent produced an 11x smaller skill bank than the ungated version. By being "picky," the agent avoids the "context clutter" that leads to performance collapse.
- Stability via Gating: Without validation, agents often suffer from "transient gains"—they look like they are learning, but they are actually overfitting to recent tasks at the expense of general capability.
- Contrastive Learning is Crucial: Learning from why a specific attempt failed compared to why another succeeded provides much richer signal than just looking at successful outcomes.
- Theoretical Certification: The authors prove that you can calculate exactly how many tasks you need in your validation set to ensure that your agent's "drawdown" (performance loss) stays below a certain threshold with high probability.
Key Results
The authors tested VALVE across three frontier models: GPT-5.5, Claude-4.6, and MiniMax-M2.7.
- Performance Gains: VALVE achieved an average final gain of 14.9 points and a peak gain of 16.5 points across all models and repositories.
- Drawdown Reduction: The validation gate reduced average drawdown by 75%. As seen in Figure 1, while an ungated agent's performance might crash after a few hundred steps, the VALVE-gated agent remains stable.
- Efficiency: The resulting skill banks were significantly more compact, making them easier to manage and cheaper to use in terms of prompt tokens.
Limitations and Future Work
- Computational Cost: Running a 50-task validation panel for every potential update is expensive in terms of both time and API tokens. The authors note that validated evolution takes roughly 2.4x longer than ungated evolution.
- Static Retriever: The current framework keeps the retrieval mechanism fixed. Future work could explore "evolving the retriever" alongside the skill bank.
- Distribution Shift: The theoretical guarantees assume tasks are drawn from a stationary distribution. If the nature of the software engineering tasks changes radically mid-stream, the fixed validation panel might become less effective.
Summary updated