ToolACE-R: Model-aware Iterative Training and Adaptive Refinement for Tool learning

Xingshan Zeng, Wei Liu, S. C. Hu, Zezhong Wang, Lingzhi Wang, Long Li, Yasheng Wang, Lifeng Shang, Xin Jiang, Ruiming Tang, Qun Liu

Abstract

Tool learning, which allows Large Language Models (LLMs) to leverage external tools for solving complex user tasks, has emerged as a promising avenue for extending model capabilities. However, existing approaches primarily focus on data synthesis for fine-tuning LLMs to invoke tools effectively, largely ignoring how to fully stimulate the potential of the model. In this paper, we propose ToolACE-R, a novel framework that includes both model-aware iterative training and adaptive refinement for tool learning. ToolACE-R features a model-aware iterative training procedure that progressively adjust training samples based on the model’s evolving capabilities to maximize its potential. Additionally, it incorporates self-refinement training corpus which emphasizes LLM's ability to iteratively refine their tool calls, optimizing performance without requiring external feedback. Furthermore, we introduce adaptive self-refinement for efficient test-time scaling, where the trained model can autonomously determine when to stop the process based on iterative self-refinement. We conduct extensive experiments across several benchmark datasets, showing that ToolACE-R achieves competitive performance compared to advanced LLMs. The performance can be further improved efficiently through adaptive self-refinement. These results highlight the effectiveness and generalizability of ToolACE-R, offering a promising direction for more efficient and scalable tool learning.

Introduction

While LLMs are increasingly used to invoke APIs and external tools, training them effectively remains a challenge. Most current research focuses on data synthesis—generating massive amounts of training examples. However, this "more is better" approach ignores a critical factor: the compatibility between the data and the specific model being trained.

If training samples are too easy, the model learns nothing new. If they are too far beyond the model's current knowledge, they can trigger hallucinations or degrade performance. Furthermore, while "scaling test-time compute" has worked for reasoning tasks, applying it indiscriminately to tool-calling is inefficient. Simple queries don't need multiple rounds of refinement, yet existing methods often apply the same computational effort to every task.

The ToolACE-R Framework

The authors introduce ToolACE-R, which consists of two primary innovations designed to align training data with model capability and optimize inference efficiency.

Model-Aware Iterative Training

Instead of using static rules to judge data quality, ToolACE-R employs a dynamic difficulty metric. This metric measures how well the model performs on a sample before training. By using the Hungarian algorithm to find the best structural alignment between predicted tool calls and ground truth, the system calculates a Jaccard-style overlap score. Training proceeds in rounds, where the framework selects "Goldilocks" data—samples that are neither trivial nor impossibly difficult for the model's current state. As the model improves, the training set's difficulty naturally shifts upward. Additionally, the training corpus is augmented with self-refinement interactions, teaching the model to fix its own mistakes based on refinement prompts.

Adaptive Self-Refine Iterative Inference

During inference, the model does not simply provide a single answer. It iteratively refines its output. Crucially, this process is adaptive: the model terminates the refinement loop as soon as it produces two identical tool-calling sequences in a row. This stability serves as an autonomous signal that the model has reached a likely correct conclusion, preventing unnecessary computation on simpler tasks.

The ToolACE-R framework consists of model-aware iterative training and adaptive self-refine iterative inference.

Experimental Setup

The researchers utilized a variety of rigorous benchmarks to evaluate ToolACE-R:

  • Training Data: A single-turn subset of the ToolACE dataset.
  • Evaluation Benchmarks: The Berkeley Function Call Leaderboard (BFCL) served as the primary benchmark, covering both synthetic and real-world scenarios. Additional tests were conducted on ACEBench, API-Bank, and ToolAlpaca to assess first-step tool-calling capabilities.

Key Results and Discussion

ToolACE-R demonstrates that specialized fine-tuning can allow smaller open-source models to punch well above their weight class.

  • BFCL Performance: ToolACE-R (8B) achieved an overall accuracy of 86.33%, surpassing GPT-4o (83.96%), DeepSeek-V3 (85.63%), and Llama-3.1-70B-Instruct (83.26%).
  • Ablation Success: Removing the "Model-Aware Data Selection" caused the biggest drop in performance (from 86.33% down to 85.04%), confirming that what the model learns is as important as how it learns.
  • Efficiency in Scaling: The average number of iterations needed on the BFCL benchmark was only 2.4 (out of a maximum of 5). This proves that adaptive refinement provides the benefits of test-time scaling without the linear cost of fixed-step refinement.
Comparison of accuracy across iterations for Vanilla vs. Adaptive Self-Refine methods.

Conclusion and Future Work

ToolACE-R offers a promising direction for more efficient and scalable tool learning. By focusing on the alignment between model capability and data difficulty, and by introducing an autonomous stopping criterion for self-refinement, it achieves state-of-the-art performance with reduced computational overhead. Future work will explore extending these techniques to complex, multi-turn dialogues and dynamically adjusting selection thresholds during the training process.

Summary updated

Abstract

Tool learning, which allows Large Language Models (LLMs) to leverage external tools for solving complex user tasks, has emerged as a promising avenue for extending model capabilities. However, existing approaches primarily focus on data synthesis for fine-tuning LLMs to invoke tools effectively, largely ignoring how to fully stimulate the potential of the model. In this paper, we propose ToolACE-R, a novel framework that includes both model-aware iterative training and adaptive refinement for tool learning. ToolACE-R features a model-aware iterative training procedure that progressively adjust training samples based on the model’s evolving capabilities to maximize its potential. Additionally, it incorporates self-refinement training corpus which emphasizes LLM's ability to iteratively refine their tool calls, optimizing performance without requiring external feedback. Furthermore, we introduce adaptive self-refinement for efficient test-time scaling, where the trained model can autonomously determine when to stop the process based on iterative self-refinement. We conduct extensive experiments across several benchmark datasets, showing that ToolACE-R achieves competitive performance compared to advanced LLMs. The performance can be further improved efficiently through adaptive self-refinement. These results highlight the effectiveness and generalizability of ToolACE-R, offering a promising direction for more efficient and scalable tool learning.
BibTeX
@inproceedings{zeng2026toolace,
  author = {Xingshan Zeng and Wei Liu and S. C. Hu and Zezhong Wang and Lingzhi Wang and Long Li and Yasheng Wang and Lifeng Shang and Xin Jiang and Ruiming Tang and Qun Liu},
  title = {{ToolACE-R: Model-aware Iterative Training and Adaptive Refinement for Tool learning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2026},
  doi = {10.1609/aaai.v40i41.40759},
}
Figure