GUARD: Stabilizing Recursive Self-Improvement with Regret Guidance

Minghe Zhang, Xuetao Kang, Yifei Ming, Yanfei Chen, Han Yu, Yufan Zhuang, Chen-Yu LeearXiv 2026 · arXiv

TL;DR

GUARD is a new framework that allows autonomous AI agents to improve their own "harness"—the collection of prompts, rules, and memories that guide them—without needing human labels or model weight updates. By decomposing tasks into atomic self-checks and using a "directional regret" signal, GUARD ensures that learning new skills doesn't break old ones, achieving stable, monotonic performance gains where previous methods often oscillated or regressed.

What Problem Does This Paper Address?

Modern AI agents aren't just raw models; they are "harnessed" by complex prompts, tool definitions, and memory systems. While we want agents to learn and refine these harnesses from experience—a process called Recursive Self-Improvement (RSI)—doing so autonomously is notoriously difficult.

Without an external "grader" or ground-truth answers, self-editing agents often fall into three traps:

  1. Local Regressions: A fix for one failure accidentally breaks a task that previously worked.
  2. Catastrophic Forgetting: As the agent adapts to new types of tasks, it overwrites the rules that made it successful at earlier ones.
  3. Memory Pollution: The harness becomes bloated with redundant or conflicting rules, slowing down the agent and wasting tokens.

Current methods often rely on simple scalar scores (e.g., "did I succeed?"). These scores are "noisy" failure signals because they conflate what an edit newly broke with what simply remains unsolved.

Methods

The authors introduce GUARD (Grader-free Unsupervised Environment Design with Adaptive Rule Distillation), a closed-loop framework for reference-free harness evolution. The core of GUARD is a four-stage loop:

  1. Self-Review: Before a task, the agent generates a "checklist" of atomic binary requirements (e.g., "Did I use the correct date format?"). After execution, a two-layer self-reviewer (sandbox execution + semantic trace analysis) grades the run against this checklist.
  2. Paired Regret (Analyze): This is the "secret sauce." GUARD compares performance across rounds to calculate directional regret. It specifically identifies which criteria were "Regressed" (newly broken), which are "Unsolved Gaps," and which are "Mastered." Mastered rules are "pinned" to prevent overwriting.
  3. Pool & Selection (Propose): Instead of trying one edit at a time, GUARD generates a parallel pool of candidate harnesses. It uses a critic-free, group-relative fitness score (inspired by the GRPO algorithm used in models like DeepSeek-V3) to pick the best candidate.
  4. Soft-HURT Gate: A safety mechanism that only promotes an edit if its gains significantly outweigh any induced "hurt" (regressions). It also uses [iff <condition>] preconditions to narrow the scope of new rules so they only trigger when relevant.

At test time, all the complex machinery (self-review, pooling, gating) is turned off. The agent simply runs with its single, optimized "frozen" harness in a single pass.

The GUARD framework: a four-stage closed loop for stable, reference-free harness evolution.

Datasets

The researchers evaluated GUARD across several challenging benchmarks:

  • Drift Gauntlet: A benchmark specifically designed to test for catastrophic forgetting across shifting task streams (e.g., moving from SQL tasks to scientific reasoning).
  • PAST-Bench: Focuses on long-term memory retention and procedural updates.
  • EvoHarnessBench: Tests the ability to evolve skills as new tools and interfaces are introduced.
  • Professional Suites: Custom-built high-stakes reasoning tasks in Legal and Finance domains.

Key Insights and Ideas

  • Generation-Verification Asymmetry: It is significantly easier for a model to verify if a finished trajectory followed a specific rule than it is to generate a perfect trajectory that follows ten rules simultaneously. GUARD exploits this by using the model as its own "checklist" architect.
  • Directional Regret over Scalar Scores: By tracking which specific sub-requirements broke, GUARD can prioritize fixing regressions over tackling long-standing gaps. This "first, do no harm" approach is what prevents the performance oscillations seen in prior work.
  • Scoping via Preconditions: Rather than deleting rules that cause regressions, GUARD learns to wrap them in [iff] blocks. This allows the agent to maintain specialized fixes for edge cases without "over-triggering" them on standard tasks.

Key Results

GUARD consistently outperformed prior state-of-the-art RSI methods like RRSI, GEPA, and Meta-Harness.

  • Monotonic Improvement: On the Drift Gauntlet benchmark, while other methods saw performance drop as they "forgot" old skills, GUARD improved at every single checkpoint, achieving a final accuracy of 73.2% (+3.3pp over the base model).
  • Zero Forgetting: GUARD achieved zero "block drops" and zero post-block forgetting across all tested streams.
  • Efficiency: Because it produces a compact, optimized harness, GUARD cut deployment token usage by 54.5% compared to the base agent.
  • Reliability: Across 19 different execution-gated promotion stages in various benchmarks, GUARD never once admitted a regressive update.
Comparison of continual harness evolution. Left: Prior methods oscillate and forget. Right: GUARD improves monotonically with zero drops.

Limitations and Future Work

  • Checklist Quality: The system's success depends on the model's ability to derive a comprehensive checklist. If the "Turn-0" checklist misses a critical task invariant, GUARD cannot optimize for it.
  • Computational Cost during Adaptation: While test-time deployment is highly efficient (O(1) context), the adaptation phase requires running multiple parallel candidates (the $M$-slot pool), which increases the initial "training" compute.
  • Future Directions: The authors suggest extending GUARD to multi-agent settings where different agents could evolve specialized harnesses and share "pinned" rules. There is also potential to apply these "text-space gradients" to help distill knowledge from larger "teacher" models into smaller, more efficient "student" agents.

Summary updated

Abstract

Autonomous AI agents pair a frozen base model with an editable harness of prompts, skills, tool rules, and memory. Recursive self-improvement (RSI) adapts this harness from experience without weight updates, yet self-editing without external graders often backfires: reflection and scalar scores conflate what an edit newly broke with what remains unsolved, causing local regressions, skill forgetting, and memory bloat. We introduce GUARD (Grader-free Unsupervised Environment Design [UED] with Adaptive Rule Distillation), a closed-loop framework for reference-free continual harness evolution. Our key insight is to decompose each task into pre-execution self-checks and guide harness updates via directional regret. Each round grades trajectories against these checks, pinning mastered rules while prioritizing newly broken behaviors, and selects among parallel candidate edits by critic-free group-relative fitness. A Soft-HURT gate then promotes an edit only when gains outweigh regressions, scoping specialized fixes to prevent over-triggering and guaranteeing non-regression on its gated buffer. At test time, one frozen harness runs in a single pass. Across benchmarks on memory, skill evolution, continual task streams, and domain reasoning, GUARD consistently improves performance without catastrophic forgetting or memory pollution while substantially cutting deployment token usage.
BibTeX
@misc{zhang2026guard,
  author = {Minghe Zhang and Xuetao Kang and Yifei Ming and Yanfei Chen and Han Yu and Yufan Zhuang and Chen-Yu Lee},
  title = {{GUARD: Stabilizing Recursive Self-Improvement with Regret Guidance}},
  year = {2026},
  eprint = {2611.23476},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.23476},
}
Figure