AgentTune: A Unified Training Lifecycle for Tool-Using LLM Agents

R E Zera Marveen Lyngkhoi, Abhivansh Gupta, Vidushee Vats, Ram Mohan Rao Kadiyala, Vinay Kumar Sankarapu, Pratinav SetharXiv 2026 · arXiv

TL;DR

AgentTune is a unified Python library designed to manage the entire lifecycle of tool-using LLM agents—from building and training to evaluation and production maintenance. By introducing a standardized "EventLog" trajectory format, it eliminates the friction between different stages of development, enabling advanced features like self-healing loops and agentic distillation.

What Problem Does This Paper Address?

Building a production-ready LLM agent is currently a fragmented process. Developers often use one library for orchestration (like LangGraph), another for reinforcement learning (like TRL), and a third for evaluation (like the lm-eval harness).

The core issue is "interface friction": the data representation of an agent's reasoning path (its trajectory) changes at every step. An RL rollout looks different from a production audit log, which looks different from a fine-tuning dataset. This forces developers to repeatedly reconstruct logic for scoring, failure detection, and loss masking. AgentTune aims to provide a "common substrate"—a single way to represent what an agent did, why it did it, and how to learn from it.

Methods

The backbone of AgentTune is the EventLog, a two-tier trajectory representation.

  • Light Tier: Observational data (what tool was called, what was the result).
  • Full Tier: Includes token spans and logprobs, essential for training.

As shown in Figure 1, this schema acts as the central hub for five distinct layers: AgentTune Architecture

AgentTune Architecture

Key technical innovations include:

  • Unified Projections: The library can project an EventLog into different formats (SFT rows, audit records, or evaluator inputs) without mutating the original data. (See Figure 3 for the specific projections).
  • Segment-Aware Loss Masking: During training, AgentTune automatically masks environment responses (tool outputs) so the model doesn't try to "memorize" the external world. It also applies different weights to reasoning vs. tool calls (Figure 4).
  • Shared RL Core: It provides a factory for TRL-backed trainers (GRPO, PPO, DPO, etc.) that natively support multi-turn tool-calling rollouts (Figure 5).
  • DECIDE Workflows: A YAML-based engine for defining production decision logic, which automatically generates audit trails for the "self-healing" loop.

Datasets

The authors demonstrate AgentTune using several specialized datasets:

  • CUAD (Contract Understanding Atticus Dataset): Used for a multi-hop legal RAG case study.
  • LegalBench-RAG: Includes MAUD, ContractNLI, and PrivacyQA for testing the transferability of search policies.
  • LexGLUE (ECTHR task): Used to evaluate agentic distillation.
  • Synthetic Data: The library includes a six-stage synthesis pipeline that turns raw document corpora into grounded, multi-hop QA datasets.

Key Insights and Ideas

  • Trajectory as the First-Class Citizen: By making the interaction history the central unit of data, the library allows an artifact to move from orchestration to RL to monitoring without custom adapters.
  • Frugality via RL: The authors found that training agents with Group Relative Policy Optimization (GRPO) to be "frugal"—using fewer searches—actually improved accuracy. This suggests that learning when to stop searching is as important as learning what to search for.
  • Procedural Distillation: AgentTune enables "agentic distillation," where a small student model (e.g., 0.5B parameters) learns the procedure of a large teacher model. By masking the teacher's retrieved text, the student focuses on learning the reasoning steps and tool-calling logic.
  • Closing the Loop: The "self-healing" mechanism is a standout feature. It detects production failures, uses an LLM to classify the root cause (e.g., "wrong tool"), generates a corrected trajectory, and triggers a gated retraining job.

Key Results

The most significant evidence comes from the legal contract search use case:

  • Performance Boost: Using GRPO, the agent's exact match (EM) score on held-out CUAD questions rose from 60.0% to 69.0%.
  • Efficiency: Simultaneously, the number of searches per question dropped from 3.00 to 2.90, proving the agent became more efficient.
  • Generalization: A policy trained on CUAD transferred successfully to unseen corpora like PrivacyQA, where the gold-document hit rate (Doc@8) improved from 0.83 to 0.98.
  • Distillation Success: On the LexGLUE ECTHR task, a 1.5B student model trained with masked trajectories achieved a 55.4 Pass@1, outperforming standard SFT (33.1) and unmasked trajectory training (49.5).

Limitations and Future Work

  • Scaling: AgentTune currently inherits the single-node scaling limits of the TRL library. It is not yet a cluster-scale system like verl or OpenRLHF.
  • CLI Maturity: While the Python SDK is robust, the command-line interface does not yet support tool-calling training.
  • Domain Specificity: Most empirical evidence is currently limited to English-language legal documents.
  • Security: Tool execution isolation is currently handled at the OS process level. The authors plan to implement more robust containerized or remote-host isolation in the future.
  • Benchmark Integration: The authors acknowledge the need to integrate with external agent benchmarks like WebArena or ToolBench for better public calibration.

Summary updated

Abstract

Tool-using agents increasingly require a common substrate for building, training, evaluating, distilling, and maintaining agentic systems, yet existing tooling typically addresses these stages in isolation. We present AgentTune, a source-available Python library that provides a unified framework for developing and optimizing tool-using agents. At its core, AgentTune introduces a two-tier EventLog representation that preserves agent trajectories while supporting consistent projections for training, evaluation, and auditing. Building on this substrate, the library provides a shared agent runtime, tool-calling reinforcement learning, agentic distillation, retrieval-augmented generation, decision workflows, evaluation utilities, and a designed path toward self-healing agents. These components are exposed through composable interfaces so that agent strategies, training methods, evaluation procedures, and recovery mechanisms can be combined without requiring separate representations or pipelines. We demonstrate the framework across agentic tasks including legal contract search and domain-specific applications, illustrating its use for trajectory-based optimization, tool-use evaluation, and cross-task agent development. AgentTune is released as a source-available library at https://github.com/Lexsi-Labs/AgentTune
BibTeX
@misc{lyngkhoi2026agenttune,
  author = {R E Zera Marveen Lyngkhoi and Abhivansh Gupta and Vidushee Vats and Ram Mohan Rao Kadiyala and Vinay Kumar Sankarapu and Pratinav Seth},
  title = {{AgentTune: A Unified Training Lifecycle for Tool-Using LLM Agents}},
  year = {2026},
  eprint = {2610.13335},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.13335},
}
Figure