TL;DR
HERA introduces a co-evolutionary framework where an LLM agent's "harness" (its surrounding code and prompts) and its training environment evolve together to improve agentic abstention—the ability to recognize and stop when a task is impossible. By iteratively targeting an agent's specific weaknesses with new, mutated environments, HERA produces a robust harness that significantly boosts reliability, transfers across diverse model families, and allows smaller models to match the performance of giants at a fraction of the cost.
What Problem Does This Paper Address?
Large Language Model (LLM) agents are becoming remarkably adept at using tools to solve multi-step problems. However, they suffer from a significant reliability gap: they don't know when to quit. When faced with an underspecified request or an environment lacking necessary prerequisites, agents often "hallucinate" progress, incurring unnecessary API costs, wasting time, or even performing unintended, irreversible actions.
This challenge is known as agentic abstention. Current approaches usually involve optimizing an agent against a fixed set of tasks. The problem is that once an agent learns to handle those specific failures, the dataset provides no further signal for improvement. HERA tackles this by treating the training environment as a moving target that evolves to expose the agent's latest vulnerabilities.
Methods
The HERA framework consists of two main stages:
- Verifiable Task Construction: The system generates diverse environments and tasks. Crucially, it uses "environment mutation" to create matched pairs. It takes a solvable task and systematically alters a prerequisite (e.g., making a specific database field unavailable) to create an infeasible version. An independent "solver" model verifies that the feasible version is solvable and the infeasible version truly requires abstention.
- Harness–Environment Co-evolution: This is an iterative loop.
- Failure Diagnosis: An LLM-based analyst examines execution trajectories to identify why an agent failed to complete a task or failed to abstain.
- Environment Evolution: An "environment designer" uses these diagnoses to create new, more challenging task pairs that specifically target the agent's current weaknesses.
- Harness Evolution: A "harness optimizer" modifies the agent's scaffold—its prompts, memory management, tool logic, and control flow—to address the diagnosed failures, all while keeping the underlying LLM fixed.
Datasets
- HERA-BENCH: A new, held-out evaluation benchmark introduced by the authors. It consists of 60 verified task pairs (120 total instances) across domains disjoint from the training set. These were manually reviewed by the authors to ensure scenario plausibility and natural language realism.
- Training Pool: The co-evolution process started with 20 task pairs and expanded to 50 pairs (100 instances) over six rounds of evolution.
Key Insights and Ideas
- The Power of the Harness: The paper demonstrates that you don't necessarily need to fine-tune the LLM itself to make it more reliable. By evolving the "harness" (the logic governing how the LLM interacts with tools and memory), you can significantly improve agent behavior.
- Logic is Transferable: Perhaps the most surprising finding is that a harness evolved using one model (GPT-5.6-Luna) provides massive reliability gains when applied to 19 other models, including open-weight models like DeepSeek and Qwen. This suggests that the logic for "when to stop" is somewhat model-agnostic.
- Evolutionary Complexity: As the co-evolution progresses, the harness logic naturally evolves from a simple executor to a sophisticated system featuring evidence-based decision gates, cross-source reconciliation, and privacy guards.
- Closing the Gap: HERA enables a "performance-cost arbitrage." A smaller, cheaper model equipped with a HERA-evolved harness can match or exceed the reliability of a much larger, state-of-the-art model using a standard harness.
Key Results
- Abstention Boost: On HERA-BENCH, the evolved harness improved abstention accuracy from 61.7% to 83.3%.
- Balanced Performance: Unlike simple prompting (which often makes agents too "lazy"), HERA improved both abstention on impossible tasks and completion rates on feasible tasks (from 68.3% to 76.7%).
- Broad Transferability: The harness improved abstention accuracy by an average of 15.3 percentage points across 19 different LLMs without any model-specific tuning.
- Cost Efficiency: GPT-5.6-Luna using the HERA harness matched the performance of the much more powerful GPT-6-Astra at an estimated 85% lower API cost.
Limitations and Future Work
- Inference Overhead: The evolved harness is more complex, leading to higher token usage and latency per turn (roughly 2-3x). However, the authors note this is often offset by the agent avoiding long, failing trajectories on infeasible tasks.
- Synthetic vs. Real-world: While HERA-BENCH was manually refined, the training data is synthetically generated. There may be "in-the-wild" failure modes that synthetic evolution doesn't capture.
- Fixed Model Weights: The study focused entirely on the harness. A promising future direction is the "tri-evolution" of the harness, the environment, and the model weights (via fine-tuning) simultaneously.
Summary updated