AskToAct: Enhancing LLMs Tool Use via Self-Correcting Clarification

Xuan Zhang, Yongliang Shen, Zhe Zheng, Linlin Wu, Wen Qi Zhang, Yuchen Yan, Qiuying Peng, Jun Wang, Weiming Lü2025 · DOI

Abstract: Xuan Zhang, Yongliang Shen, Zhe Zheng, Linjuan Wu, Wenqi Zhang, Yuchen Yan, Qiuying Peng, Jun Wang, Weiming Lu. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.

2025.

Introduction

Most research into tool-augmented Large Language Models (LLMs) operates under an idealistic assumption: that user queries are always explicit and unambiguous. In real-world applications, however, user requests are often messy, providing incomplete information (e.g., "Book me a ride" without specifying a destination) or ambiguous intents. When faced with these "unspecified queries," current LLMs frequently hallucinate missing parameters or fail to complete the task.

Existing attempts to solve this via interactive clarification—where the model asks the user for more information—face two major hurdles: data scarcity and error accumulation. Creating high-quality datasets for clarification requires expensive human annotation. Furthermore, if a model makes a single mistake in a multi-turn dialogue, it often lacks the training to recover, leading to task failure. ASKTOACT is a novel framework designed to address these challenges by automating data generation and introducing a robust self-correction mechanism.

The ASKTOACT Framework

ASKTOACT tackles the limitations of current tool-use models through two primary innovations: automated reverse-engineered data generation and self-correction training.

1. Automated Data Generation

Instead of relying on human-written vague queries, the researchers developed a method to systematically "break" existing datasets of perfect queries. By identifying the parameters required for an API call (e.g., location and food_type) and stripping them from the original user query, they created a "vague" query with a known "ground truth" of what information is missing. This approach allowed for the automated generation of over 35,000 high-quality training examples.

2. Self-Correction Training

To prevent models from failing after an initial error, ASKTOACT intentionally injects errors into training dialogues. The framework identifies five common error types, such as Redundant Clarification (asking for information already provided) and Irrelevant Clarification. The model is trained using special tokens (<SOE> and <EOE>) to demarcate these errors. Through selective masking during fine-tuning, the model learns to recognize its mistakes, apologize, and provide the correct response without learning to repeat the erroneous behavior.

The ASKTOACT framework pipeline, illustrating unspecified query generation, clarification dialogue construction, and self-correction training.

Experimental Setup

The study utilized two primary datasets modified for Intent Clarification (IC):

  • xlam-IC: Derived from the xlam-function-calling-60k dataset, containing 29,821 training samples across 21 functional categories.
  • Taskbench-IC: Used for Out-of-Domain (OOD) testing to evaluate the model's ability to generalize to entirely unseen APIs.

Results and Analysis

The experimental results demonstrate that ASKTOACT significantly enhances the robustness and efficiency of LLMs in tool-use scenarios. A fine-tuned Qwen2.5-7B model using the ASKTOACT strategy achieved performance comparable to or better than much larger models like GPT-4o.

Main experimental results comparing ASKTOACT against closed-source and open-source LLMs across intent clarification and tool invocation metrics.

Key Findings:

  • Intent Recovery: ASKTOACT recovered over 57% of critical unspecified intents, a significant improvement over baseline models.
  • Efficiency: Clarification efficiency improved by an average of 10.46%, requiring fewer turns to reach the correct tool call.
  • Accuracy: The framework achieved over 81% tool selection accuracy and 68% parameter resolution accuracy.
  • Generalization: In OOD testing, the model successfully handled unseen APIs, proving it learned the underlying logic of clarification rather than simple memorization.
  • Human Satisfaction: In real-world testing, ASKTOACT variants achieved a user satisfaction score of 4.61/5, compared to 3.80/5 for the base model.

Conclusion and Future Work

ASKTOACT demonstrates that specialized data strategies and self-correction mechanisms can enable smaller models to outperform larger counterparts in complex tool-use tasks. By treating tool parameters as anchors for user intent, the framework provides a mathematically grounded way for models to interact with users.

Future work will focus on expanding the diversity of error patterns beyond the five pre-defined types and exploring context-dependent queries where intent must be inferred from the results of previous API calls. Additionally, researchers aim to better account for human factors like user patience in long-form dialogues.

Summary updated

BibTeX
@misc{zhang2025asktoact,
  author = {Xuan Zhang and Yongliang Shen and Zhe Zheng and Linlin Wu and Wen Qi Zhang and Yuchen Yan and Qiuying Peng and Jun Wang and Weiming Lü},
  title = {{AskToAct: Enhancing LLMs Tool Use via Self-Correcting Clarification}},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-main.682},
}
Figure