Evidence-Governed Adoption of Self-Generated Updates in Autonomous AI Agents

Subodh LonkararXiv 2026 · arXiv

TL;DR

This paper introduces Evidence-Governed Evolution (EGE), a statistical framework for safely adopting self-generated updates in autonomous AI agents. By combining fresh audit data, candidate-specific evaluator calibration, and non-inferiority testing, EGE ensures that agent modifications—like new prompts or tool configurations—provide genuine improvements without silently degrading core capabilities or falling prey to evaluator bias.

What Problem Does This Paper Address?

Autonomous agents are increasingly capable of "self-evolution," modifying their own prompts, memory schemas, and workflows based on past interactions. However, letting an agent update its own persistent state is risky. The authors identify four main challenges:

  1. Adaptive Overfitting: A candidate update might perform well on recent tasks simply because it was tuned to them, failing to generalize.
  2. Evaluator Optimism: Agents often use LLMs to evaluate their own performance. These internal judges can be biased or "manipulable" by changes in the agent's output style.
  3. Capability Drift: An update that improves a specific task might inadvertently break a "protected" capability, like reliability or safety constraints.
  4. Operational Risk: Statistical success on a benchmark doesn't guarantee safety under production load or prevent irreversible external side effects.

Methods

EGE moves away from "greedy" adoption toward a layered, audit-based approach. The key components are:

  • Separation of Concerns: Candidate generation is strictly separated from the confirmatory audit. Once a candidate is proposed, it is "frozen" to prevent further tuning on audit data.
  • Evaluator Calibration: Instead of trusting an LLM-based evaluator blindly, EGE uses a small, independent calibration sample to compare the operational evaluator against a "reference procedure" (e.g., a human expert or a deterministic verifier). This accounts for candidate-specific biases.
  • Simultaneous Statistical Bounds: The framework applies Hoeffding bounds and union-bound allocations to provide a simultaneous lower confidence bound across a countable sequence of updates and multiple metrics.
  • Reference Lineage & Bridge Tests: To prevent the "moving baseline" problem, EGE formalizes how to replace a reference model. It uses "bridge tests" to explicitly calculate and bound the cumulative degradation allowed over a versioned lineage of references.
  • Two-Phase Commitment: Acceptance isn't just a statistical event. A candidate must pass the offline audit and then survive a bounded canary deployment (monitoring for tool errors, latency, etc.) before being persistently committed to the agent's state.
Table 1: Conceptual positioning of EGE relative to representative research streams. Entries describe primary emphasis, not an exhaustive claim that a method cannot be combined with another control.

Datasets

The paper does not evaluate a specific deployed agent on standard benchmarks. Instead, it utilizes a synthetic Monte Carlo stress test designed to expose statistical failure modes.

  • Monte Carlo Sensitivity Study: 3,000 independent campaigns were simulated to test how the framework handles optimistic evaluator bias ($b \in \{0, 0.10, 0.25\}$) and varying sample sizes ($n, m \in \{5k, 20k, 80k\}$).
  • Calibration-Mismatch Test: A deliberately misspecified condition where the calibration sample does not represent the actual audit bias, used to demonstrate the importance of representative sampling.

Key Insights and Ideas

  • Calibration must be candidate-specific: An update might change an agent's output style in a way that makes the internal evaluator more optimistic. EGE argues that a global calibration factor is insufficient; you must calibrate for each frozen candidate.
  • The "Statistical Price" of Progress: Theorem 2 formalizes the cost of changing your baseline. Every time you update your reference, you "spend" some of your safety margin (bridge tolerance), and EGE makes this accumulation explicit.
  • Inconclusive is a valid outcome: EGE distinguishes between a "Reject" (the update is bad) and "Inconclusive" (we don't have enough data to prove it's good). This prevents the system from adopting updates based on noise.
  • Governance is not a metric: The framework treats mandatory governance checks (like tool permissions) as hard constraints that cannot be offset by high performance scores in other areas.

Key Results

  • Theorem 1 (Simultaneous Coverage): The authors prove that their construction provides valid coverage for a sequence of adaptively generated candidates, ensuring the probability of a "false acceptance" stays below a global error budget $\alpha$.
  • Stress Test Performance: In the Monte Carlo study, when the evaluator was highly optimistic ($b=0.25$), a "naive" adoption rule accepted harmful updates in 99.9% of campaigns. In contrast, the calibrated EGE rule had zero harmful acceptances across all correctly specified conditions.
  • Sample Size Requirements: The paper provides an audit-planning inequality. For a target effect gap of 0.10, the framework might require ~84k total observations for the first candidate to guarantee power, highlighting that rigorous self-evolution is "sample-intensive."

Limitations and Future Work

  • Independence Assumption: The current statistical guarantees rely on independent observations. Real-world agent tasks are often clustered (e.g., by user session), which requires more complex concentration inequalities for dependent data.
  • Reference Validity: EGE only controls discrepancy relative to a reference procedure. If the human expert or verifier used for calibration is fundamentally flawed, the guarantees are undermined.
  • Computational Cost: The requirement for fresh, candidate-specific calibration and audit samples is expensive. Future work is needed to find more sample-efficient ways to achieve uniform calibration over candidate classes.
  • Irreversible Actions: While EGE can roll back a configuration, it cannot undo external actions (like a sent email or a database deletion). The authors emphasize that EGE must be part of a broader security architecture.

Summary updated

Abstract

Autonomous agents modify prompts, memories, tools, workflows, and policies using interaction histories. Yet a plausible modification is not necessarily suitable for persistent adoption: candidate selection is adaptive, evaluators may be imperfect, and local gains can hide degradation against an earlier baseline. We introduce Evidence-Governed Evolution (EGE), a framework for governing adoption of self-generated agent modifications. EGE separates candidate generation from confirmatory adoption, evaluates frozen candidates on fresh paired audits, requires meaningful improvement on a prespecified primary objective, protects declared capabilities through reference-relative non-inferiority tests, and applies governance constraints before bounded canary deployment. We establish an evaluator-calibrated simultaneous lower-bound construction for a countable sequence of adaptively generated candidates. It combines candidate-wise Hoeffding bounds with independent calibration samples comparing the operational evaluator with a prespecified reference procedure. We also formalise reference replacement through bridge tests, derive a cumulative degradation bound along a versioned reference lineage, and provide an audit-planning condition relating effect-size margin, sample sizes, and acceptance probability. A Monte Carlo stress test illustrates calibration under candidate-dependent evaluator optimism; it is a stylised statistical experiment, not an evaluation of a deployed agent. The guarantees are conditional on bounded paired scores, fresh sampling, a valid reference-evaluation construct, and declared evaluation distributions; they do not establish universal agent safety or robustness to arbitrary distribution shift.
BibTeX
@misc{lonkar2026evidence,
  author = {Subodh Lonkar},
  title = {{Evidence-Governed Adoption of Self-Generated Updates in Autonomous AI Agents}},
  year = {2026},
  eprint = {2611.16378},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.16378},
}
Figure