ToolACE-R: Tool Learning with Adaptive Self-Refinement

Xingshan Zeng, Weiwen Liu, Xu Huang, Zezhong Wang, Lingzhi Wang, Liangyou Li, Yasheng Wang, Lifeng Shang, Xin Jiang, Ruiming Tang, Qun Liu

Abstract

Tool learning, which allows Large Language Models (LLMs) to leverage external tools for solving complex user tasks, has emerged as a promising avenue for extending model capabilities. However, current approaches primarily focus on data synthesis for fine-tuning LLMs to invoke tools effectively, largely ignoring how to fully stimulate the potential of the model. In this paper, we propose ToolACE-R, a novel method that introduces adaptive self-refinement for tool invocations. Our approach features a model-aware iterative training procedure that progressively incorporates more training samples based on the model's evolving capabilities. Additionally, it allows LLMs to iteratively refine their tool calls, optimizing performance without requiring external feedback. To further enhance computational efficiency, we integrate an adaptive mechanism when scaling the inference time, enabling the model to autonomously determine when to stop the refinement process. We conduct extensive experiments across several benchmark datasets, showing that ToolACE-R achieves competitive performance compared to advanced API-based models, even without any refinement. Furthermore, its performance can be further improved efficiently through adaptive self-refinement. Our results demonstrate the effectiveness of the proposed method, which is compatible with base models of various sizes, offering a promising direction for more efficient tool learning.

Introduction

While Large Language Models (LLMs) are increasingly capable of invoking external tools like APIs and search engines, two major hurdles persist in current research. First, there is a significant data mismatch in fine-tuning; static synthetic datasets often fail to challenge the model appropriately, either being too simple to foster growth or too complex, leading to hallucinations. Second, inefficient test-time scaling remains a challenge. While self-refinement—where a model checks and fixes its own output—is powerful, current methods often apply a fixed number of steps to every query, wasting computation on simple tasks.

ToolACE-R addresses these issues by "stimulating the model's intrinsic potential" through a framework that aligns training data with evolving model capabilities and introduces a smarter, adaptive way to scale computation during inference.

Methodology

ToolACE-R introduces three core technical innovations designed to optimize both training and inference:

  1. Model-Aware Difficulty Metric: Instead of static heuristics, difficulty is defined by the model's actual performance. Using a Jaccard-style overlap and the Hungarian algorithm to align predicted tool calls with ground truth, the framework selects "Goldilocks" samples—those that are informative but not insurmountable—for the next training iteration.
  2. Self-Refinement Data Augmentation: Training sets are augmented with multi-turn refinement dialogues. Crucially, this includes "identity cases" where the model's initial answer is correct. This teaches the model that refinement does not always necessitate change.
  3. Adaptive Self-Refinement (Inference): During testing, the model iteratively refines its output. Because it was trained on identity cases, it learns to output the same string when confident. The system stops when two consecutive iterations produce identical results, providing an autonomous stopping criterion.
Overview of the ToolACE-R framework, showing the model-aware iterative training and adaptive self-refinement inference loops.

Key Results and Insights

ToolACE-R, built on a Llama-3.1-8B backbone, achieved remarkable results across standard benchmarks:

  • Outperforming Frontier Models: On the Berkeley Function Call Leaderboard (BFCL), ToolACE-R reached an overall accuracy of 86.33%, surpassing GPT-4o (83.96%) and DeepSeek-V3 (85.63%).
  • Massive Gains over Base Models: Compared to the base Llama-3.1-8B-Instruct, ToolACE-R improved accuracy on ACEBench from 50.63% to 84.00%.
  • Efficiency via Adaptivity: On the BFCL benchmark, the model averaged only 2.4 iterations (out of a max of 5). This proves that adaptive refinement provides the benefits of test-time scaling without the linear cost increase of fixed-step methods.
Comparison of accuracy across iterations for Vanilla vs. Adaptive Self-Refine, demonstrating the stability and superior performance of the adaptive approach.

As shown in the figure above, the adaptive mechanism prevents the performance fluctuations often seen in vanilla self-refinement, where the model might "fix" a correct answer into an incorrect one. By learning when to stop, ToolACE-R maintains a higher and more stable accuracy plateau.

Conclusion

ToolACE-R demonstrates that model-aware data selection and adaptive inference mechanisms can significantly boost tool-calling performance. The framework is backbone-agnostic, showing consistent improvements across different model families like Qwen2.5 and Mistral. Future work may explore integrating external tool execution feedback to further enhance the refinement loop and testing the framework on larger model scales beyond 8B parameters.

Summary updated

Abstract

Tool learning, which allows Large Language Models (LLMs) to leverage external tools for solving complex user tasks, has emerged as a promising avenue for extending model capabilities. However, current approaches primarily focus on data synthesis for fine-tuning LLMs to invoke tools effectively, largely ignoring how to fully stimulate the potential of the model. In this paper, we propose ToolACE-R, a novel method that introduces adaptive self-refinement for tool invocations. Our approach features a model-aware iterative training procedure that progressively incorporates more training samples based on the model's evolving capabilities. Additionally, it allows LLMs to iteratively refine their tool calls, optimizing performance without requiring external feedback. To further enhance computational efficiency, we integrate an adaptive mechanism when scaling the inference time, enabling the model to autonomously determine when to stop the refinement process. We conduct extensive experiments across several benchmark datasets, showing that ToolACE-R achieves competitive performance compared to advanced API-based models, even without any refinement. Furthermore, its performance can be further improved efficiently through adaptive self-refinement. Our results demonstrate the effectiveness of the proposed method, which is compatible with base models of various sizes, offering a promising direction for more efficient tool learning.
BibTeX
@inproceedings{zeng2026toolace,
  author = {Xingshan Zeng and Weiwen Liu and Xu Huang and Zezhong Wang and Lingzhi Wang and Liangyou Li and Yasheng Wang and Lifeng Shang and Xin Jiang and Ruiming Tang and Qun Liu},
  title = {{ToolACE-R: Tool Learning with Adaptive Self-Refinement}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2026},
  doi = {10.1609/aaai.v40i41.40759},
}
Figure