ToolReflection: Improving Large Language Models for Real-World API Calls with Self-Generated Data

Gregory Polyakov, Ilseyar Alimova, Dmitry Abulkhanov, Ivan Sedykh, Andrey Bout, Sergey Nikolenko, Irina Piontkovskaya2025 · DOI

Abstract

Gregory Polyakov, Ilseyar Alimova, Dmitry Abulkhanov, Ivan Sedykh, Andrey Bout, Sergey Nikolenko, Irina Piontkovskaya. Proceedings of the 1st Workshop for Research on Agent Language Models (REALM 2025).

2025.

Introduction

While Large Language Models (LLMs) have demonstrated significant capability in tool use, they often struggle when transitioning from controlled laboratory environments to real-world APIs. In practical applications, tool documentation is frequently incomplete, APIs return cryptic error messages, and parameter requirements are often implicit. Current models typically fail to treat these errors as constructive feedback, often "hallucinating" parameters or failing to recover after an initial rejection. ToolReflection addresses this gap by teaching LLMs to self-correct their API calls through a trial-and-error loop driven by real-time execution feedback.

The ToolReflection Pipeline

The core innovation of this work is the ToolReflection pipeline, which shifts the training paradigm from simple instruction tuning to reflection-based learning. This process involves four critical stages:

  1. Error Collection: A pretrained model is deployed against various tool-use queries to harvest instances of failure, such as incorrect parameter types or missing fields.
  2. Feedback Integration: The system captures the raw error message from the Python interpreter or HTTP response.
  3. Correction Generation: The researchers construct "error-correction pairs" that include the initial failed call, the error feedback, a rationale (thought process) explaining the failure, and the subsequent corrected tool call.
  4. Fine-Tuning: The model is fine-tuned on this augmented data, internalizing a meta-strategy for debugging: identifying the error, reasoning about its cause, and adjusting the strategy accordingly.
The ToolReflection process: from initial query and failed execution to self-correction based on runtime feedback.

Experimental Setup and Datasets

The authors evaluated ToolReflection using established benchmarks like ToolAlpaca, ToolBench, and GPT4Tools, while introducing several new evaluation sets to test robustness:

  • GPT4Tools-OOD & OOD-Hard: These sets feature five open-source APIs (e.g., flight schedules, nutrition analysis) entirely absent from the training data. The "Hard" variant omits tool usage examples, forcing the model to rely on raw reasoning.
  • Multistep-100: A dataset designed for complex, interdependent tool chains where the output of one API serves as the input for the next.
  • GPT4FakeTools: A synthetic dataset of 3,180 samples used to diversify tool signatures during the training phase.

Key Results

The ToolReflection method yielded substantial performance gains, particularly in challenging out-of-distribution (OOD) scenarios where static documentation is most likely to fail. The most striking improvement was observed in the GPT4Tools-OOD-Hard benchmark, where the success rate (SR) increased by 56.2% over the baseline. In standard OOD tasks, the model saw a 25.4% boost.

Success rates across Seen, Unseen, OOD, and Hard OOD tasks, highlighting the significant performance leap of ToolReflection in difficult scenarios.

Beyond these headline figures, the model demonstrated a remarkable ability to recover from initial failures, correcting approximately 51% of calls that initially returned an error in the GPT4Tools + GPT4FakeTools setup. These results suggest that ToolReflection teaches a generalizable meta-strategy for tool interaction rather than just memorizing specific API signatures.

Discussion and Limitations

The research highlights that in real-world tool use, a model's ability to recover from failure is often more critical than its initial accuracy. However, several challenges remain. The current implementation focuses on single-step corrections; in complex multi-step chains, errors may originate from mistakes made several steps prior, requiring more sophisticated backtracking mechanisms. Additionally, the method's efficacy remains dependent on the quality of the error feedback provided by the API. Future work will explore more efficient, unsupervised data generation and the integration of preference learning (e.g., DPO or RLHF) to further refine self-correction logic.

Summary updated

BibTeX
@misc{polyakov2025toolreflection,
  author = {Gregory Polyakov and Ilseyar Alimova and Dmitry Abulkhanov and Ivan Sedykh and Andrey Bout and Sergey Nikolenko and Irina Piontkovskaya},
  title = {{ToolReflection: Improving Large Language Models for Real-World API Calls with Self-Generated Data}},
  year = {2025},
  doi = {10.18653/v1/2025.realm-1.14},
}
Figure