EvoSkill-S1: Evolving Reusable Skills with Genetic-Inspired Operators

Yujun Wang, Jiqiang Li, Yinuo Lu, Jinhe Bi, Aniri, Wenxuan Ye, Boliang Liu, Sikuan Yan, Shuning Wang, Xuebing Zhou, S{ö}ren Pirk, Hinrich Sch{ü}tze, Yunpu MaarXiv 2026 · arXiv

TL;DR

EvoSkill-S1 is a novel reinforcement learning (RL) framework that treats reusable problem-solving skills as objects of evolution. By applying genetic-inspired operators—like mutation and crossover—to a "skill bank," the system iteratively transforms weak strategies into robust, abstract procedural guidance, leading to significant performance boosts in complex mathematical reasoning and code generation.

What Problem Does This Paper Address?

While Large Language Model (LLM) agents have become adept at reasoning, they often struggle with "hard-negative" failures—problems they consistently miss despite having access to previous experiences. Most current methods for maintaining "skill banks" (collections of strategies and tips) are passive: they simply append new experiences or use basic heuristics to summarize them. This often results in shallow, redundant, or overly specific updates that don't generalize well.

The authors of EvoSkill-S1 argue that skills shouldn't just be accumulated; they should be evolved. The challenge lies in how to systematically repair brittle strategies, combine complementary partial skills, and ensure that evolved skills remain abstract enough to be useful for unseen problems.

Figure 1: Heuristic skill-bank updates versus direct skill- level evolution in EVOSKILL-S1, where an RL-trained evolver transforms existing skills through genetic- inspired operators.

Methods

The EvoSkill-S1 pipeline moves beyond simple reflection by implementing a dedicated, RL-trained Skill Evolver.

  1. Iterative Skill Evolution: The framework identifies "hard negatives" (problems the agent failed). It retrieves "parent" skills from the bank and uses the Skill Evolver to generate "child" skills.
  2. Genetic-Inspired Operators: Instead of unconstrained reflection, the evolver uses five specific operators:
  3. Revision: Small, conservative repairs to a strategy.
  4. Cross-revision: Borrowing a specific "motif" or safeguard from a related problem.
  5. Mutation: Introducing a completely alternative algorithmic approach.
  6. Crossover: Combining the "backbone" of one parent with a local improvement from another.
  7. Dual-reflection: Strengthening correctness-critical safeguards.
  8. RL Training (GRPO): The evolver is trained using Group Relative Policy Optimization (GRPO). It learns to favor child skills that actually improve the performance of a fixed base solver on downstream tasks.
  9. Skill Bank Fusion: Since different operators solve different types of failures, the framework fuses operator-specific banks into a single, high-quality collection.
  10. Procedural Skill Interface (PSI): To prevent evolved skills from being too tied to their source problems, the PSI "proceduralizes" them—stripping away specific constants and stories to create abstract, retrieval-ready guidance.
Figure 2: Overview of the EVOSKILL-S1 pipeline. An initial skill bank is evolved through fixed-operator runs, where an RL-trained evolver generates candidate child skills and evaluator-guided write-back updates the bank. Operator-specific banks are then fused, and the resulting evolved skills are accessed through the Procedural Skill Interface at inference time.

Datasets

The authors evaluated EvoSkill-S1 across a rigorous suite of math and code benchmarks:

  • Math: AIME24, AIME25, AMC23, Minerva Math, and OlympiadBench.
  • Code: APPS (test set) and LiveCodeBench.
  • Training/Evolution Pools: Hard subsets mined from DAPO (math) and APPS-train (code), totaling roughly 4,000 math and 2,200 code problems.

Key Insights and Ideas

  • Skills as Genetic Material: Treating skills as entities that can be mutated and recombined is more effective than treating them as static diary entries of past failures.
  • The Power of RL in Evolution: Training the evolver with RL (GRPO) significantly increases the "yield" of useful skills. As shown in Figure 4, RL-trained evolvers solve more hard negatives and produce more high-quality "write-back" candidates than those using vanilla LLM prompting.
  • Operator Complementarity: No single genetic operator is a silver bullet. Some failures require the "conservative repair" of Revision, while others require the "radical shift" of Mutation. This justifies the "Fusion" approach.
  • Abstraction is Mandatory: Evolution often produces skills cluttered with details from the specific problem that triggered the update. The "Proceduralization" step is critical for ensuring these skills actually help when the agent encounters a brand-new problem.

Key Results

EvoSkill-S1 consistently outperformed both initialized skill banks and strong external baselines like Reflexion and ExpeL-SI.

  • Performance Gains: The framework achieved relative improvements of up to 12.39% over the initial skill banks.
  • Superiority over Baselines: Compared to the strongest external baseline without direct skill evolution, the relative improvement reached as high as 20.39%.
  • Efficiency: The RL-based training allowed the system to recover more hard negatives under a fixed "write-back" budget (the number of new skills allowed into the bank).
Figure 4: Effect of RL training on skill-evolution ef- ficiency. Solved once counts hard negatives solved at least once during evolution, while written-back children counts child skills accepted into the bank.

Limitations and Future Work

  • Fixed Solver: Currently, the base solver (the model doing the actual math/coding) remains frozen. The authors suggest that co-evolving the solver and the skill bank simultaneously could unlock even higher performance.
  • Computational Overhead: While the evolution happens "offline," the process of generating and evaluating multiple candidate child skills for every failure is computationally intensive.
  • Domain Expansion: The current work focuses on math and code, where "correctness" is easily verifiable. Applying this to more subjective domains (like creative writing or general planning) would require more complex reward modeling.

Summary updated

Abstract

Reusable skills allow language-model agents to accumulate and reuse problem-solving experience. However, most existing methods improve a skill bank only indirectly through rollout-driven accumulation and heuristic reflection, often resulting in shallow and weakly controlled updates. We present EvoSkill-S1, an RL-driven framework that instead treats reusable skills themselves as the direct objects of evolution. Given hard-negative failures, EvoSkill-S1 selects parent skills and transforms them into candidate child skills through five genetic-inspired operators: revision, cross-revision, mutation, dual-reflection, and crossover. Downstream task rewards train the skill evolver to generate more effective child skills, while evaluator-guided write-back selectively retains beneficial updates. Complementary Skill Bank Fusion consolidates operator-specific banks to exploit their distinct recovery patterns, while a Procedural Skill Interface supports transfer to unseen problems. Across all reported math and code benchmarks, EvoSkill-S1 improves the corresponding initialized skill banks under both Qwen and GPT-family solvers, with relative gains of up to 12.39%. Compared with the strongest external baseline without direct skill evolution, the largest relative improvement reaches 20.39%.
BibTeX
@misc{wang2026evoskill,
  author = {Yujun Wang and Jiqiang Li and Yinuo Lu and Jinhe Bi and Aniri and Wenxuan Ye and Boliang Liu and Sikuan Yan and Shuning Wang and Xuebing Zhou and S\{ö\}ren Pirk and Hinrich Sch\{ü\}tze and Yunpu Ma},
  title = {{EvoSkill-S1: Evolving Reusable Skills with Genetic-Inspired Operators}},
  year = {2026},
  eprint = {2611.22610},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.22610},
}
Figure