HarnessForge: Joint Harness and Policy Evolution for Adaptive Agent Systems

Mingju Chen, Can Lv, Guibin Zhang, Heng Chang, Shiji ZhouarXiv 2026 · arXiv

TL;DR

LLM agents often struggle when switching between different task environments because their external execution structures (harnesses) and internal reasoning models (policies) are optimized in isolation. This paper introduces HarnessForge, a meta-adaptive framework that co-evolves the harness and policy as a coupled pair. By closing this "compatibility gap," HarnessForge achieves up to a 12% performance boost over state-of-the-art baselines while remaining highly rollout-efficient.

What Problem Does This Paper Address?

When we build Large Language Model (LLM) agents, we typically deploy them into highly diverse environments—some require multi-step tool use, others demand heavy data retrieval, and some need strict state tracking. To handle this, an agent system relies on two core parts: an external harness (the system prompts, memory modules, and tool protocols) and an internal policy (the LLM's reasoning behavior).

Traditionally, developers optimize these components separately. Search-style methods find the best workflow structure while keeping the model frozen, whereas training-style methods fine-tune the model under a fixed workflow. The authors argue that this creates a severe compatibility gap (as illustrated in Figure 1). A highly sophisticated harness might fail if the model isn't trained to follow its complex rules; conversely, a powerful model remains bottlenecked by a rigid, poorly designed workflow.

Figure 1: A three-stage view of LLM agent adaptation: fixed handcrafted systems, local component adaptation, and harness–policy co-evolution (HarnessForge).

Methods

HarnessForge solves this by formulating the agent system as a coupled harness–policy pair and introducing a multi-round, meta-adaptive joint evolution workflow.

Figure 2: Overview of the HarnessForge co-evolution workflow. Starting from a harness–policy pair, each round diagnoses execution failures, tailors the harness over planning, action, and memory modules, trains a harness- conditioned adapter from curated trajectories, and selects improved matched pairs for the next round.

Each evolutionary round consists of two main phases:

  1. Fault-Guided Harness Tailoring: The agent executes a batch of training tasks. If it fails, a meta-agent (GPT-5.5) inspects the failure trajectories to perform fault attribution—pinpointing whether planning, action, or memory caused the error. It then generates revised harness candidates, filtering them using a budget-efficient Pareto-selection mechanism based on accuracy, token cost, and latency.
  2. Harness-Conditioned Policy Alignment: Instead of training a universally stronger model, HarnessForge trains a lightweight, harness-specific LoRA adapter to align the model with the newly tailored harness. Crucially, this step reuses the successful trajectories already generated during the harness evaluation phase, making it "rollout-neutral" (requiring zero extra environment interactions).

Datasets

The framework utilizes a 3,800-task training pool spanning complex environment interactions (EnvScaler-RL), general tool use (ToolHop), and offline retrieval (Wikipedia-based QA).

Evaluation is conducted across five distinct benchmarks:

  • ToolHop & API-Bank: For multi-hop tool selection and structured API execution.
  • RestBench-TMDB: A movie-database REST API environment.
  • SearchQA (combining HotpotQA and 2WikiMultiHopQA): For retrieval-heavy, multi-hop question answering.

Key Insights and Ideas

  • Co-Specialization is Essential: The authors ran cross-combination experiments pairing harnesses and policies from different evolutionary rounds. As shown in the compatibility matrix (Figure 4), mismatched pairs underperformed significantly. The best results come from matched pairs, proving that the model must specialize to the exact nuances of its execution interface.
  • Rollout-Neutral Training Cuts Costs: Reinforcement learning for agents is notoriously expensive due to the massive number of environment interactions required. HarnessForge bypasses this by recycling successful rollouts from the harness-testing phase to train the policy adapters, maximizing data efficiency.
  • Task-Specific Pressures Vary: The framework revealed that different task regimes stress different modules. API-heavy tasks heavily pressure the action layer (demanding strict format contracts and schema preflights), while multi-hop QA tasks pressure the planning layer to maintain long-term intent across several steps.
Figure 4: Harness–policy compatibility matrices on API-Bank. Rows denote evolved harnesses and columns denote evolved policy across evolution rounds.

Key Results

  • Consistent State-of-the-Art Performance: HarnessForge consistently outperformed both harness-only search methods (e.g., ADAS, AFlow) and policy-only training methods (e.g., SFT, GRPO, RLOO), averaging a +3.56% gain across all metrics using Qwen3-4B and Qwen3-8B backbones.
  • Massive Gains on TMDB: The most pronounced improvement occurred on the RestBench-TMDB benchmark, where HarnessForge achieved a 12.0% absolute success rate boost over the strongest baseline.
  • Superior Pareto Efficiency: Budget-performance analysis demonstrated that HarnessForge sits comfortably on or near the Pareto frontier, delivering performance comparable to heavy on-policy RL methods at a fraction of the rollout cost.

Limitations and Future Work

  • Scale of Backbones: The study primarily focused on smaller, resource-constrained models (4B and 8B parameters). Whether the same magnitude of compatibility gains applies to frontier-scale models (like GPT-4 or Claude 3.5 Sonnet) remains an open question.
  • Constrained Search Space: The harness evolution relies on a structured, prompt-driven edit space over predefined modules (Planning, Action, Memory) rather than allowing arbitrary code-level rewrites or inventing entirely new tool abstractions.
  • Long-Horizon Overhead: Even with rollout reuse, running iterative evolution rounds in exceptionally long-horizon environments can still become computationally expensive. The authors suggest exploring proxy evaluations or early-stopping criteria in future work.

Summary updated

Abstract

LLM agents are increasingly expected to operate across heterogeneous task regimes that require distinct execution paradigms. This challenges fixed agent systems and motivates system-level meta-adaptation beyond isolated component updates. While existing works have adapted external harness or trained underlying reasoning policies, full-system adaptation remains insufficiently characterized. The adaptation space between structure and execution is rarely made explicit, and the compatibility between the external harness and the internal reasoner is not optimized jointly. We propose HarnessForge, a meta-adaptive framework for evolving LLM agent systems. HarnessForge formulates an agent system as a harness--policy pair, defining a stable adaptation space that separates harness-level execution structure from policy-level reasoning behavior. It then performs harness--policy co-evolution through fault-guided harness tailoring and harness-conditioned policy alignment. Experiments across five benchmarks from diverse domains show that HarnessForge consistently improves both Qwen3-4B and Qwen3-8B backbones, outperforming harness-only and policy-only baselines with gains of up to 12.0\% over the strongest baseline and achieving favorable rollout-efficiency tradeoffs, demonstrating that harness--policy co-evolution is effective, and that executable compatibility between the harness and reasoning policy is essential for agent-system adaptation. The code is available at https://github.com/mingju-c/HarnessForge.
BibTeX
@misc{chen2026harnessforge,
  author = {Mingju Chen and Can Lv and Guibin Zhang and Heng Chang and Shiji Zhou},
  title = {{HarnessForge: Joint Harness and Policy Evolution for Adaptive Agent Systems}},
  year = {2026},
  eprint = {2606.01779},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2606.01779},
}

Loading related content...

Figure