TL;DR
This paper investigates whether AI agents can truly improve by analyzing their own "failure traces" (records of past mistakes). While giving a teacher model access to these traces does provide a modest boost in isolated decision-making, about 60% of the total improvement comes from simple prompt optimization that never even sees the agent's errors.
What Problem Does This Paper Address?
The AI community is currently fascinated by "self-evolving" agents—systems that supposedly get better over time by reflecting on their own performance. However, most of these agents have "frozen weights" (their core neural network doesn't change), meaning any "learning" must happen in the "harness": the system prompts, notes, and control logic surrounding the model.
The authors tackle a critical question: Is the gain we see in these systems actually coming from the content of the failures, or is it just the result of a better search for effective prompts? If a teacher model can improve an agent just as well without seeing its mistakes, then "learning from experience" might be an overstatement.
Methods
The researchers used a Qwen3.5-9B model as their agent and a larger "teacher" model to write "lessons" (formatted as WHEN situation THEN action) to guide it.
The core innovation is a rigorous experimental design featuring:
- No-Evidence Search (The Control): A teacher model tries to improve the agent's prompt using the same budget and feedback loop, but without seeing any of the agent's actual interaction traces. This isolates the "added value" of the failure data.
- The Gate: A hierarchical Bayesian screening process that only admits lessons if they show a credible gain across multiple contexts without causing "harm" (regressions) elsewhere.
- Blind Scorer Audit: To ensure results weren't skewed by benchmark errors, independent reviewers audited the ground truth of the test sets.
Datasets
The study evaluated agents on two primary benchmarks:
- $\tau^2$-bench retail: A customer-service domain with complex policies. It was tested at the "decision level" (isolated choices) and the "conversation level" (full multi-turn interactions).
- BFCL (Berkeley Function Calling Leaderboard) multi-turn: A benchmark for tool-use and API calling.
Key Insights and Ideas
- Search is the Silent Winner: A significant portion of what looks like "learning from experience" is actually just the teacher model finding better ways to phrase existing rules. In their retail tests, search alone achieved 60% of the total improvement.
- The Generalization Gap: Lessons that help an agent make a single correct decision often fail to improve—or even hurt—performance in full, fluid conversations.
- Benchmarks are Brittle: The authors found that a staggering 60.8% of the held-out instances in the BFCL benchmark were "defective" (where a correct response would be scored as wrong). This highlights a massive "noise floor" that can make small improvements look like significant breakthroughs.
- Generic is Often Better: In some cases, a simple "neutral note" (e.g., "Handle each request using the functions as needed") outperformed complex, learned lessons.
Key Results
The quantitative findings suggest that while failure traces help, their impact is limited:
- Decision Accuracy: On isolated retail decisions, trace-based lessons beat the no-evidence search by +4.0 percentage points. While statistically significant, this was below the 5-point threshold the authors pre-registered as "worthwhile."
- Conversation Success: In full retail conversations, the learned lessons showed no demonstrated improvement over the standard baseline.
- BFCL Performance: No trace-written lessons passed the rigorous "Gate" for the tool-use benchmark. The one lesson that did pass (from the no-evidence search) failed to replicate its gains on held-out data.
The headline results are summarized in the table below:
Limitations and Future Work
The authors are careful to note several limitations:
- Scope: The study focused on a single 9B model and one family of teacher models.
- Rule Discovery vs. Phrasing: On the retail benchmark, the teacher already knew the target rules. The experiment measured how well the teacher could phrase those rules for the agent, not whether it could discover new rules from scratch.
- Snapshot Decisions: Decision-level tests are "snapshots" that don't allow the agent to recover from errors, which might not reflect real-world agentic behavior.
The paper concludes with a call for more rigorous evaluation in the field: researchers should always report a matched "no-evidence" search control and perform blind audits of their scorers to ensure they aren't just measuring noise.
Summary updated