TL;DR
LLM agents that self-improve by reflecting on their failures often get "distracted" by noisy signals, such as broken environments or impossible tasks. GEPA-Curator introduces an agentic harness that filters out these unhelpful experiences and retrieves past successful trajectories to provide a clean, contrastive learning signal. This approach leads to performance gains of up to 21.2% over state-of-the-art prompt optimizers like GEPA and ACE.
What Problem Does This Paper Address?
Recursive self-improvement (RSI) is the "holy grail" of agentic AI: the idea that an agent can interact with an environment, reflect on its mistakes, and update its own prompt or code to perform better next time. However, existing reflective optimizers (like GEPA) operate on a dangerous assumption: that every failure is the prompt's fault.
In reality, agentic failures are often "noisy." An agent might fail because a website is temporarily down, a tool is missing from the harness, or the task itself is ambiguous. When a "Reflector" LLM tries to learn from these cases, it often misattributes the failure to a prompt deficiency. This leads to "negative transfer"—where the agent learns harmful, overgeneralized strategies (like exhaustive searching when a simple direct link would suffice) that actually degrade performance on solvable tasks.
Methods
GEPA-Curator acts as an intelligent "outer loop" around existing prompt optimizers. Instead of feeding every rollout to the optimizer, it employs two key mechanisms:
- Task Curation (Section 3.1): The Curator maintains a per-task history of trajectories and outcomes. It analyzes this history to decide which tasks are worth running in the next training batch. It prioritizes "optimizable" tasks while temporarily skipping those it diagnoses as "Mastered" (already solved reliably), "Impossible" (failures due to environment/harness limits), or "Flaky" (inconsistent results).
- Prior Success Retrieval (Section 3.2): When a task fails, the Curator searches its history for a past successful trajectory of that same task. It provides this success to the Reflector as a "trusted reference." This enables contrastive learning: the Reflector can see exactly where the current attempt diverged from a known-good path, rather than just guessing how to fix a failure in a vacuum.
To make this practical, the authors also developed an Efficient GEPA-Curator variant that uses "skip intervals" (skipping a task for multiple epochs once diagnosed) and "compact histories" to minimize LLM token costs.
Datasets
The authors evaluated GEPA-Curator across a diverse range of agentic environments:
- WebArena (GoBrowse): Realistic web tasks across GitLab, Shopping, Shopping Admin, and Reddit.
- Agentic Benchmarks: ALFWorld (text-based house chores), AppWorld (API-based tasks), and $\tau$-bench (retail tool-use).
- QA Benchmarks: AIME 2025, IFBench, HoVer, and HotpotQA (to test performance on cleaner, non-agentic signals).
Key Insights and Ideas
- Noise is Poison: The authors proved that mixing "Impossible" or "Flaky" tasks into a training set significantly caps the maximum accuracy an optimizer can reach. As shown in Figure 2, learning curves for "Optimizable" tasks peak much lower when contaminated by noisy signals.
- Curation is Dynamic: A key "aha" moment is that skipping isn't permanent. A task that was "Impossible" for a weak prompt might become solvable as the prompt improves. The Curator re-evaluates skipped tasks every epoch to see if they should be reintroduced.
- Contrastive Learning is Powerful: Providing a prior success (an "anchor") was found to be critical. It prevents the Reflector from hallucinating fixes for failures caused by external noise.
- Beyond Prompts: The methodology is general enough to improve Meta-Harness, an optimizer that modifies the agent's underlying Python code rather than just its prompt.
Key Results
GEPA-Curator consistently outperformed baselines across different optimizer models (Gemini 3.1 Pro and Claude 3.5 Opus):
- WebArena: Achieved a 3.4% to 5.0% absolute gain over standard GEPA, representing a 1.54x to 1.78x improvement in the "optimization gain" over the initial seed prompt.
- Agentic Benchmarks: Showed average gains of 14.3% over GEPA and 5.0% over ACE using Gemini. With Claude, it matched or exceeded GEPA on all sets, with a massive 21.2% gain over ACE on some tasks.
- Cost Efficiency: The "Efficient" variant maintained nearly all the performance gains while only increasing total optimization costs by 1.31x compared to the baseline.
Limitations and Future Work
- Signal Cleanliness: The authors noted that curation provides less benefit on structured QA datasets (like math problems) where the evaluation signal is already very clean.
- Curator Quality: The system's success depends on the Curator LLM's ability to accurately diagnose failures. If the Curator is too aggressive in skipping tasks, it might miss optimization opportunities.
- Future Directions: The paper suggests extending this agentic curation to more complex "playbook" generation and multi-agent systems where the interaction noise is even higher.
For more details, you can find the full paper on arXiv under ID 2610.37651v1.
Summary updated