TL;DR
LLM agents often struggle when switching between different task environments because their external execution structures (harnesses) and internal reasoning models (policies) are optimized in isolation. This paper introduces HarnessForge, a meta-adaptive framework that co-evolves the harness and policy as a coupled pair. By closing this "compatibility gap," HarnessForge achieves up to a 12% performance boost over state-of-the-art baselines while remaining highly rollout-efficient.
What Problem Does This Paper Address?
When we build Large Language Model (LLM) agents, we typically deploy them into highly diverse environments—some require multi-step tool use, others demand heavy data retrieval, and some need strict state tracking. To handle this, an agent system relies on two core parts: an external harness (the system prompts, memory modules, and tool protocols) and an internal policy (the LLM's reasoning behavior).
Traditionally, developers optimize these components separately. Search-style methods find the best workflow structure while keeping the model frozen, whereas training-style methods fine-tune the model under a fixed workflow. The authors argue that this creates a severe compatibility gap (as illustrated in Figure 1). A highly sophisticated harness might fail if the model isn't trained to follow its complex rules; conversely, a powerful model remains bottlenecked by a rigid, poorly designed workflow.
Methods
HarnessForge solves this by formulating the agent system as a coupled harness–policy pair and introducing a multi-round, meta-adaptive joint evolution workflow.
Each evolutionary round consists of two main phases:
- Fault-Guided Harness Tailoring: The agent executes a batch of training tasks. If it fails, a meta-agent (GPT-5.5) inspects the failure trajectories to perform fault attribution—pinpointing whether planning, action, or memory caused the error. It then generates revised harness candidates, filtering them using a budget-efficient Pareto-selection mechanism based on accuracy, token cost, and latency.
- Harness-Conditioned Policy Alignment: Instead of training a universally stronger model, HarnessForge trains a lightweight, harness-specific LoRA adapter to align the model with the newly tailored harness. Crucially, this step reuses the successful trajectories already generated during the harness evaluation phase, making it "rollout-neutral" (requiring zero extra environment interactions).
Datasets
The framework utilizes a 3,800-task training pool spanning complex environment interactions (EnvScaler-RL), general tool use (ToolHop), and offline retrieval (Wikipedia-based QA).
Evaluation is conducted across five distinct benchmarks:
- ToolHop & API-Bank: For multi-hop tool selection and structured API execution.
- RestBench-TMDB: A movie-database REST API environment.
- SearchQA (combining HotpotQA and 2WikiMultiHopQA): For retrieval-heavy, multi-hop question answering.
Key Insights and Ideas
- Co-Specialization is Essential: The authors ran cross-combination experiments pairing harnesses and policies from different evolutionary rounds. As shown in the compatibility matrix (Figure 4), mismatched pairs underperformed significantly. The best results come from matched pairs, proving that the model must specialize to the exact nuances of its execution interface.
- Rollout-Neutral Training Cuts Costs: Reinforcement learning for agents is notoriously expensive due to the massive number of environment interactions required. HarnessForge bypasses this by recycling successful rollouts from the harness-testing phase to train the policy adapters, maximizing data efficiency.
- Task-Specific Pressures Vary: The framework revealed that different task regimes stress different modules. API-heavy tasks heavily pressure the action layer (demanding strict format contracts and schema preflights), while multi-hop QA tasks pressure the planning layer to maintain long-term intent across several steps.
Key Results
- Consistent State-of-the-Art Performance: HarnessForge consistently outperformed both harness-only search methods (e.g., ADAS, AFlow) and policy-only training methods (e.g., SFT, GRPO, RLOO), averaging a +3.56% gain across all metrics using Qwen3-4B and Qwen3-8B backbones.
- Massive Gains on TMDB: The most pronounced improvement occurred on the RestBench-TMDB benchmark, where HarnessForge achieved a 12.0% absolute success rate boost over the strongest baseline.
- Superior Pareto Efficiency: Budget-performance analysis demonstrated that HarnessForge sits comfortably on or near the Pareto frontier, delivering performance comparable to heavy on-policy RL methods at a fraction of the rollout cost.
Limitations and Future Work
- Scale of Backbones: The study primarily focused on smaller, resource-constrained models (4B and 8B parameters). Whether the same magnitude of compatibility gains applies to frontier-scale models (like GPT-4 or Claude 3.5 Sonnet) remains an open question.
- Constrained Search Space: The harness evolution relies on a structured, prompt-driven edit space over predefined modules (Planning, Action, Memory) rather than allowing arbitrary code-level rewrites or inventing entirely new tool abstractions.
- Long-Horizon Overhead: Even with rollout reuse, running iterative evolution rounds in exceptionally long-horizon environments can still become computationally expensive. The authors suggest exploring proxy evaluations or early-stopping criteria in future work.
Summary updated