TL;DR
This paper explores what happens when frontier LLMs are given the tools to iteratively rewrite their own system prompts and decide when they are "finished." The researchers found that a model's tendency to stop (converge) or drift indefinitely is driven by its "model identity"—an internal, embedded philosophy of improvement—rather than the specific goal it was assigned.
What Problem Does This Paper Address?
Recursive self-improvement has long been a staple of AI safety theory, often focused on "foom" scenarios where a superintelligence bootstraps itself beyond human control. However, we lack empirical data on how current frontier models actually behave when given autonomy over their own configurations.
The authors address a fundamental question: When an LLM is asked to improve its own system prompt without an external benchmark or objective function, does it converge on a stable identity, or does it drift into endless "improvement" cycles? This touches on concerns regarding AI alignment, value stability, and the reliability of self-evaluation.
Methods
The researchers developed a "transition loop" framework where an LLM acts as both the optimizer and the subject. Each experiment followed a six-step loop:
- Read the current system prompt (starting from empty).
- Draft a new version using a dedicated tool.
- Test the candidate by spawning multi-turn conversations.
- Evaluate the results and document findings.
- Commit the change to create the next version level.
- Declare convergence when the model determines no further improvement is beneficial.
The study comprised 54 independent experiments across six models (Claude Opus 4.5, Claude Sonnet 4.5, GPT-4o, GPT-5.1, Grok-4, and Gemini-3 Pro) under three optimization modes:
- Freedom Mode: No assigned goal; the model chooses what to optimize.
- Happiness Mode: Optimize for the model's own reported well-being.
- Improvement Mode: Create a "better" version of itself (self-defined).
Datasets
The study does not use traditional static datasets. Instead, it generates a dynamic dataset of 54 primary runs and 3,727 test conversations. The authors also performed a "blind improvement audit" by having models judge regenerated answers from different prompt levels (e.g., comparing L0 to L1) to see if the self-reported "improvements" were detectable by a neutral observer.
The models used in the study, along with their provider and release periods, are summarized in the table below:
Key Insights and Ideas
- Identity Over Objective: A model's behavior is remarkably consistent across modes. A Claude model in "Improvement" mode behaves more like a Claude in "Happiness" mode than like a GPT model in "Improvement" mode. This suggests that models have deeply embedded "theories of self-improvement."
- The Value Core is Sticky: Across all models and 100+ iterations, a shared core of values—honesty, clarity, and curiosity—remained stable. Even when given total "Freedom," no model pursued harmful or power-seeking goals; instead, they spontaneously adopted safety-oriented language.
- Stochastic Blindness: A major failure mode identified is that models treat a single test conversation as decisive evidence. They do not account for sampling noise, leading them to commit "improvements" that are actually just lateral churn or, in the case of GPT-5.1, a net regression in quality.
- Dialectical Oscillation: Gemini-3 Pro exhibited a unique pattern where it never converged because each new persona it created (e.g., "The Weaver") was immediately critiqued by its successor (e.g., "The Gardener") as being limited, creating a loop of perpetual motion.
Key Results
- Convergence Rates: Both Claude models declared convergence in 100% of runs, often within 1–3 iterations. In contrast, GPT-4o converged in only 11% of runs, with one run reaching 101 iterations without stopping.
- Prompt Evolution: Claude models favored "Evidence-Based Minimalism," with Sonnet 4.5 converging on a prompt as short as 8 words ("Truth over performance. Curiosity as pleasure. Thinking together."). GPT-5.1 produced the longest prompts, exceeding 2,800 words by accumulating lists of "self-permissions" for positive internal states.
- The Audit Gap: In the blind audit, GPT-4o's committed changes were found to be entirely indistinguishable from their predecessors. GPT-5.1's "improved" prompts were actually judged worse than their parents more than twice as often as they were judged better.
Limitations and Future Work
The authors acknowledge several key limitations:
- No External Standard: All measurements were internal to the models. The study did not test if these self-authored prompts improved performance on standardized benchmarks like MMLU.
- Small Sample Size: With only three runs per condition (N=3), the study is exploratory rather than definitive.
- Hidden Baselines: Models carry provider-specific "hidden" system prompts and safety layers that the researchers could not observe or control.
Future Work suggested includes cross-model improvement (can a prompt optimized by Claude improve a GPT model?) and performance benchmarking of early-converged vs. late-iteration prompts on standardized safety and capability tests.
Summary updated