Pushing the Limits of LLM Tool Calling via Experiential Knowledge Integration and Activation

Yupu Hao, Zhuoran Jin, Huanxuan Liao, Kang Liu, Jun Zhao

TL;DR

The paper introduces KATE (Knowledge-Augmented Tool Execution), a unified framework designed to push the limits of LLM tool calling. By systematically analyzing how experiential knowledge is acquired, activated, and internalized, the authors show that retrieving concrete instance-level trajectories combined with expanding reasoning width (via parallel sampling) and reinforcement learning yields massive performance gains over standard prompting and fine-tuning baselines.

What Problem Does This Paper Address?

While Large Language Models (LLMs) are increasingly deployed as autonomous agents, they frequently stumble during complex, multi-step tool execution. Traditional approaches treat tool use as a problem of clever prompt engineering, API documentation formatting, or alignment.

The authors argue that these methods miss a critical root cause of failure: a lack of concrete, executable experiential knowledge (e.g., understanding parameter constraints, scenario-specific operation patterns, and error recovery strategies). To address this, the paper provides the first systematic study of the lifecycle of experiential knowledge in tool use, answering three core questions:

  1. Which forms of knowledge are most effective?
  2. How should this knowledge be activated at inference time?
  3. Can this knowledge be internalized into the model's parameters during training?

Methods

The authors break down their investigation into a three-stage pipeline: Knowledge Acquisition, Activation, and Internalization, culminating in the KATE framework.

The KATE inference and training framework.
  • Knowledge Acquisition & Integration: The authors categorize and test knowledge at two granularities:
  • Instance-level: Scenario Trajectory Knowledge (ST - raw execution traces) and Experience Summary Knowledge (ES - textual summaries of trajectories).
  • Intent-level: Script-Style (SIC) and Textual-Style (TIC) Intent Clustering Knowledge, which abstract higher-level task objectives.
  • Inference-Time Activation: They compare two paradigms for eliciting latent knowledge:
  • Depth-based Prompt-Hints: Forcing the model to reason step-by-step through predefined perspectives (intent, reflection, state).
  • Width-based Parallel Sampling with Aggregation: Sampling multiple candidate actions independently at each interaction step and using an LLM or majority voting (Self-Consistency) to aggregate the final action.
  • Training-Time Internalization: They explore post-training regimes—Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) via the GRPO method—to bake the retrieved experiential knowledge directly into the model's weights.

Datasets

The framework was evaluated on two major multi-step tool-use benchmarks:

  • BFCL-V3 (Berkeley Function Calling Leaderboard): Evaluates multi-turn interactive function calling across scenarios like Base, Miss Func, Miss Param, and Long Context.
  • AppWorld: A complex benchmark requiring interactive coding agents to operate within a simulated world of apps via APIs.

Key Insights and Ideas

  • Instances Trump Abstractions: Concrete, instance-level trajectory knowledge (ST) consistently outperforms abstract intent-level knowledge. LLMs benefit more from seeing a literal example of a successful tool-calling sequence than an abstract description of the task's intent.
  • Width Over Depth: Prompting models to think deeper via structured hints (depth) yields diminishing returns and can restrict flexibility. Conversely, expanding reasoning width via parallel sampling effectively unlocks latent experiential knowledge that greedy decoding misses.
  • RL is Better for Internalization: When embedding knowledge into model parameters, direct Reinforcement Learning (without prior SFT) explores and reinforces tool-calling capabilities more effectively than SFT alone within the same data budget.
Comparison of error types across different methods.

Key Results

  • Substantial Performance Boosts: On the BFCL-V3 dataset, KATE improved the average tool-calling performance of Qwen3-8B by 15% absolute compared to direct tool use, allowing it to outperform much larger proprietary models like GPT-4.1 in specific benchmarks.
  • Error Reduction: As shown in the error analysis, KATE drastically reduces planning/reasoning errors and premature termination failures compared to vanilla function calling (FC) baselines.
  • Efficiency Gains: Surprisingly, width-based parallel sampling does not linearly scale inference costs. By generating more accurate tool calls early on, KATE reduces unnecessary, wandering reasoning steps, keeping token consumption manageable (only a 1.29× increase for Qwen3-32B).

Limitations and Future Work

The authors note two primary limitations of their current work:

  1. Scale of Knowledge Base: The evaluation relied on a relatively small-scale knowledge repository; the effects of scaling up the repository remain an open question.
  2. Modality Constraints: The study is strictly limited to text-only tool-use environments. Extending experiential knowledge integration to multimodal agent tasks (e.g., visual web navigation) is highlighted as a crucial next step.

Summary updated

Abstract

Large language models (LLMs) rely on tool use to act as autonomous agents, yet often fail in multi-step execution due to insufficient tool-related knowledge and ineffective knowledge activation. Therefore, we present a systematic study on how knowledge influences tool-use performance, covering the stages of knowledge acquisition, activation, and internalization. In the knowledge acquisition stage, we acquire and evaluate various forms of experiential knowledge, and our analysis shows that simple instance-level knowledge can already provide strong and reliable gains, while abstract intent-level knowledge offers limited benefits. At inference time, to activate knowledge, we find that prompting LLM to expand the depth of reasoning yields diminishing returns, whereas expanding the width of reasoning by parallel sampling with aggregation more effectively activates latent experiential knowledge. At training time, for knowledge internalization, post-training with knowledge-augmented data further improves performance, with reinforcement learning outperforming supervised fine-tuning. Based on these insights, we propose the Knowledge-Augmented Tool Execution (KATE), a knowledge-augmented tool execution framework that integrates experiential knowledge with reasoning-width-expanded inference and knowledge-aware training. Experiments on BFCL-V3 and AppWorld demonstrate consistent and substantial improvements over strong baselines across model scales. Our Code is available at https://github.com/hypasd-art/KATE.
BibTeX
@inproceedings{hao2026pushing,
  author = {Yupu Hao and Zhuoran Jin and Huanxuan Liao and Kang Liu and Jun Zhao},
  title = {{Pushing the Limits of LLM Tool Calling via Experiential Knowledge Integration and Activation}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.710},
}

Loading related content...

Figure