TL;DR
This paper introduces a novel method for training LLM agents to delegate subtasks with precise, least-privilege permissions. By leveraging the procedural knowledge embedded in "Agent Skills," the researchers synthesized a high-quality dataset of verified trajectories that teach agents exactly which tools and files a subagent needs—and which it doesn't—significantly improving both security and task success.
What Problem Does This Paper Address?
As we move from single-turn chatbots to complex multi-agent systems, "delegation" has become a core capability. A lead agent must break down a task and spin up subagents to handle specific parts. However, this introduces a critical authorization challenge: How much access should a subagent get?
If a lead agent grants access to the entire workspace and all available tools, it violates the principle of least privilege, creating security risks. Conversely, if permissions are too restrictive, the subagent will fail. Current training methods rely on execution trajectories, which show what an agent did access, but not necessarily what it needed to access. This "supervision gap" makes it difficult for models to learn the fine line between "enough" and "too much" access.
Methods
The authors propose a Skill-driven data synthesis method that turns procedural knowledge into executable supervision. The pipeline consists of three main steps:
- Task and SOP Synthesis: Starting with an "Agent Skill" (a package of code and instructions), the system generates a concrete task and a Step-by-Step Procedure (SOP). This SOP explicitly lists the tools and files required for each step.
- Agent Execution and Rollout: A "teacher" agent executes the task in a controlled environment. Crucially, the teacher uses the SOP to make explicit authorization decisions for subagents.
- Verification and Data Filtering: An independent LLM judge reviews the resulting trajectories. It checks if the task was completed correctly, if permissions were respected, and if the process felt "natural." Only successful, high-quality trajectories are kept.
Finally, the model is fine-tuned (SFT) on this data. During training, the SOP is removed, forcing the model to learn how to infer the correct permissions directly from the task description and the environment.
Datasets
The researchers generated a new dataset of 4,413 trajectories, ultimately retaining 2,871 verified examples after filtering. The dataset covers a wide range of domains, with Software Engineering, Data Analytics, and Security being the largest contributors.
For evaluation, they used:
- ClawArena-Team: A benchmark specifically designed for subagent management and resource allocation.
- Terminal-Bench 2.1 & SkillsBench: Used to ensure the method remains competitive in general task execution.
Key Insights and Ideas
- Skills as Supervision: The "aha moment" is realizing that Agent Skills are more than just tools; they are structured repositories of procedural knowledge. They provide a "ground truth" for what a task actually requires.
- SOP as a Training Scaffold: The Step-by-Step Procedure acts as a temporary scaffold. It guides the teacher agent to create perfect training data, but is discarded so the student agent learns to think for itself.
- Authorization is Task-Conditioned: The paper demonstrates that permissions shouldn't be static. A subagent's "modality" (e.g., being an image analyst vs. a legal analyst) should dictate its specific toolset and file access.
- Verification is Non-Negotiable: Ablation studies showed that removing the LLM judge (verification) actually made the model perform worse than the base model, as it began to imitate failed or over-privileged trajectories.
Key Results
The method showed dramatic improvements on the ClawArena-Team benchmark for the Qwen3.5-35B-A3B model:
- Tool Permission Precision (TPP): Increased by 13.4 percentage points.
- Workspace Permission Precision (WPP): Increased by 24.1 percentage points.
- Task Completion Rate (TCR): Improved by 11.6 percentage points.
A qualitative case study (Figure 3) highlights the difference: while a base model might grant a subagent access to an entire directory of satellite imagery, the trained model grants access only to the two specific scene files and metadata required for the subtask.
Limitations and Future Work
While the results are impressive, the authors note a few areas for further exploration:
- Judge Sensitivity: While they validated their LLM judge against GPT-4 (achieving 91% agreement), the quality of the final dataset is heavily dependent on the judge's reasoning capabilities.
- Complexity of Delegation: The current work focuses on a lead agent delegating to a single layer of subagents. Future work could explore multi-level hierarchies where subagents delegate further down the chain.
- Broader Resource Types: The study primarily looks at file and tool permissions; extending this to network access, API quotas, or financial budgets for agents would be a logical next step.
Summary updated