InfCycle: Learning to Use Tools via Inference Compute and Cycle Consistency

xiaobo liang, wenjing Xie, Juntao Li, Wanfu Wang, Yibin Chen, Min ZhangOpenReview 2025 · Submission to ICLR 2025 · Withdrawn · OpenReview

Its authors withdrew this paper, so there is no full text to read or summarize.

Abstract

The scaling of inference-time computation in large language models (LLMs) has emerged as a promising approach for enhancing reasoning capabilities by trading off inference-time and pre-training compute. The practice of how to enable LLMs to utilize additional computation at test time to improve response accuracy is crucial for both academia and industry. \textit{Proposer-Verifier}, as a typical paradigm of inference scaling, often fails to generalize to various scenarios. Specifically, in tool use tasks, LLMs face the risk of lacking effective verifiers, leading to error accumulation in multiple reasoning steps. In this work, we address these challenges by introducing \textbf{InfCycle}, a multi-stage data synthesis strategy that employs LLMs as data synthesis and employs cycle consistency verification to ensure high-quality trajectory generation. This approach utilizes step-wise cycle consistency among synthesized trajectories for a given tool, providing effective process supervision that has advantages over outcome supervision. Extensive experiments on multiple tool-use and reasoning tasks demonstrate that InfCycle efficiently enables self-improvement. It outperforms state-of-the-art baselines on StableToolBench, achieving a 75.4\% pass rate and a 79.6\% win rate using small size models (7B), without relying on external supervision or expert trajectories for warm-up.
BibTeX
@misc{liang2025infcycle,
  author = {xiaobo liang and wenjing Xie and Juntao Li and Wanfu Wang and Yibin Chen and Min Zhang},
  title = {{InfCycle: Learning to Use Tools via Inference Compute and Cycle Consistency}},
  year = {2025},
  url = {https://openreview.net/forum?id=CD2wgg9RQD},
  howpublished = {OpenReview},
  note = {Submitted to ICLR 2025},
}

Loading related content...

Figure