TL;DR
The paper introduces KATE (Knowledge-Augmented Tool Execution), a unified framework designed to push the limits of LLM tool calling. By systematically analyzing how experiential knowledge is acquired, activated, and internalized, the authors show that retrieving concrete instance-level trajectories combined with expanding reasoning width (via parallel sampling) and reinforcement learning yields massive performance gains over standard prompting and fine-tuning baselines.
What Problem Does This Paper Address?
While Large Language Models (LLMs) are increasingly deployed as autonomous agents, they frequently stumble during complex, multi-step tool execution. Traditional approaches treat tool use as a problem of clever prompt engineering, API documentation formatting, or alignment.
The authors argue that these methods miss a critical root cause of failure: a lack of concrete, executable experiential knowledge (e.g., understanding parameter constraints, scenario-specific operation patterns, and error recovery strategies). To address this, the paper provides the first systematic study of the lifecycle of experiential knowledge in tool use, answering three core questions:
- Which forms of knowledge are most effective?
- How should this knowledge be activated at inference time?
- Can this knowledge be internalized into the model's parameters during training?
Methods
The authors break down their investigation into a three-stage pipeline: Knowledge Acquisition, Activation, and Internalization, culminating in the KATE framework.
- Knowledge Acquisition & Integration: The authors categorize and test knowledge at two granularities:
- Instance-level: Scenario Trajectory Knowledge (ST - raw execution traces) and Experience Summary Knowledge (ES - textual summaries of trajectories).
- Intent-level: Script-Style (SIC) and Textual-Style (TIC) Intent Clustering Knowledge, which abstract higher-level task objectives.
- Inference-Time Activation: They compare two paradigms for eliciting latent knowledge:
- Depth-based Prompt-Hints: Forcing the model to reason step-by-step through predefined perspectives (intent, reflection, state).
- Width-based Parallel Sampling with Aggregation: Sampling multiple candidate actions independently at each interaction step and using an LLM or majority voting (Self-Consistency) to aggregate the final action.
- Training-Time Internalization: They explore post-training regimes—Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) via the GRPO method—to bake the retrieved experiential knowledge directly into the model's weights.
Datasets
The framework was evaluated on two major multi-step tool-use benchmarks:
- BFCL-V3 (Berkeley Function Calling Leaderboard): Evaluates multi-turn interactive function calling across scenarios like Base, Miss Func, Miss Param, and Long Context.
- AppWorld: A complex benchmark requiring interactive coding agents to operate within a simulated world of apps via APIs.
Key Insights and Ideas
- Instances Trump Abstractions: Concrete, instance-level trajectory knowledge (ST) consistently outperforms abstract intent-level knowledge. LLMs benefit more from seeing a literal example of a successful tool-calling sequence than an abstract description of the task's intent.
- Width Over Depth: Prompting models to think deeper via structured hints (depth) yields diminishing returns and can restrict flexibility. Conversely, expanding reasoning width via parallel sampling effectively unlocks latent experiential knowledge that greedy decoding misses.
- RL is Better for Internalization: When embedding knowledge into model parameters, direct Reinforcement Learning (without prior SFT) explores and reinforces tool-calling capabilities more effectively than SFT alone within the same data budget.
Key Results
- Substantial Performance Boosts: On the BFCL-V3 dataset, KATE improved the average tool-calling performance of Qwen3-8B by 15% absolute compared to direct tool use, allowing it to outperform much larger proprietary models like GPT-4.1 in specific benchmarks.
- Error Reduction: As shown in the error analysis, KATE drastically reduces planning/reasoning errors and premature termination failures compared to vanilla function calling (FC) baselines.
- Efficiency Gains: Surprisingly, width-based parallel sampling does not linearly scale inference costs. By generating more accurate tool calls early on, KATE reduces unnecessary, wandering reasoning steps, keeping token consumption manageable (only a 1.29× increase for Qwen3-32B).
Limitations and Future Work
The authors note two primary limitations of their current work:
- Scale of Knowledge Base: The evaluation relied on a relatively small-scale knowledge repository; the effects of scaling up the repository remain an open question.
- Modality Constraints: The study is strictly limited to text-only tool-use environments. Extending experiential knowledge integration to multimodal agent tasks (e.g., visual web navigation) is highlighted as a crucial next step.
Summary updated