An Evaluation Mechanism of LLM-based Agents on Manipulating APIs

Bing Liu, Jianxiang Zhou, Dan Meng, Haonan LuEMNLP 2024 · OpenReview

Abstract

LLM-based agents can greatly extend the abilities of LLMs and thus attract sharply increased studies. An ambitious vision – serving users by manipulating massive API-based tools – has been proposed and explored. However, we find a widely accepted evaluation mechanism for generic agents is still missing. This work aims to fill this gap. We decompose tool use capability into seven aspects and form a thorough evaluation schema. In addition, we design and release an instruction dataset and a toolset – the two sides that the agents bridge between – following the principle of reflecting real-world challenges. Furthermore, we evaluate multiple generic agents. Our findings can inspire future research in improving LLM-based agents and rethink the philosophy of API design.

Introduction

We are currently in the era of the "AI Agent," where Large Language Models (LLMs) act as the "brains" capable of using external tools (APIs) to perform complex tasks like booking flights or editing images. However, evaluating these agents is notoriously difficult. Most current benchmarks focus on end-to-end "pass/fail" performance, which fails to identify why an agent failed—whether it misunderstood the user, picked the wrong tool, or failed to parse technical output. Furthermore, many existing benchmarks use "closed-world" toy examples that do not reflect the messy reality of real-world APIs with complex dependencies and verbose documentation.

This work introduces a comprehensive evaluation framework designed to peek inside the "black box" of LLM-based agents. By decomposing tool-use capability into distinct decision-making behaviors and providing a new benchmark of 66 APIs and over 1,000 instructions, we reveal critical gaps in current agent capabilities, particularly in complex planning and user clarification.

Methodology

The core of our evaluation framework is the dissection of the agent's workflow into seven key decision behaviors:

  1. Planning: Breaking a complex task into logical subtasks.
  2. Deciding whether to use tools: Determining if internal knowledge suffices or if an API is required.
  3. Selecting useful tools: Identifying the correct API from a large candidate set.
  4. Requesting missing info: Recognizing when user input is insufficient and asking for clarification.
  5. Formulating tool calls: Correcting formatting and parameters to match API specifications.
  6. Responding to tool returns: Interpreting technical JSON results into natural language.
  7. Responding with intrinsic knowledge: Maintaining general chatbot capabilities after tool-specific fine-tuning.
The detailed workflow of an LLM-based agent, illustrating the decision-making process from user instruction to final response.

To test these behaviors, we developed a "Super APP" environment where the agent serves as a bridge between the user and a platform of real-world tools.

Datasets and Environment

We released two major resources to facilitate this evaluation:

  • The Toolset: Comprising 66 APIs organized into 27 tools across scenarios like travel, image processing, and daily life. Notably, 83% of these are sourced from RapidAPI, incorporating real-world quirks such as API dependencies (e.g., requiring a "City ID" before fetching weather data).
  • The Instruction Dataset: A collection of 1,028 instructions categorized into five types (Type I to V), ranging from simple queries to high-level tasks requiring multi-step planning and handling of "trick" questions.

Results and Analysis

We evaluated several leading models, including GPT-4, GPT-3.5, and the Qwen-1.5 series (7B, 14B, 72B).

Radar chart comparing the performance of different LLMs across eight specific evaluation tasks.

Key Findings

  • The "Clarification" Gap: While models like Qwen are proficient at tool selection, they struggle to identify missing information. Instead of asking for clarification, they often hallucinate parameters.
  • The Planning Bottleneck: Handling API dependencies (Task 7) remains the "final boss" for agents. GPT-4 achieved a 57% progress rate, while Qwen-72B managed only 26%. Smaller models (7B and 14B) failed this task entirely.
  • Tool Selection Proficiency: Most models performed well in basic selection. GPT-4 reached 97% accuracy, and even the mid-sized Qwen-14B achieved 82%.
  • Decision Reliability: GPT-4 and Qwen-72B showed high reliability in deciding when to use tools (Macro-F1 > 0.96), whereas smaller models frequently misused tools.

Conclusion

Our evaluation highlights that while LLM-based agents are improving, significant hurdles remain in complex planning and interactive clarification. The gap between "talking" (planning) and "doing" (execution) is still wide. Furthermore, the verbosity of current API documentation suggests a need for "LLM-friendly" API design standards. Future work will focus on multi-turn interactions and the challenges of large-scale API retrieval.

Summary updated

Abstract

LLM-based agents can greatly extend the abilities of LLMs and thus attract sharply increased studies. An ambitious vision – serving users by manipulating massive API-based tools – has been proposed and explored. However, we find a widely accepted evaluation mechanism for generic agents is still missing. This work aims to fill this gap. We decompose tool use capability into seven aspects and form a thorough evaluation schema. In addition, we design and release an instruction dataset and a toolset – the two sides that the agents bridge between – following the principle of reflecting real-world challenges. Furthermore, we evaluate multiple generic agents. Our findings can inspire future research in improving LLM-based agents and rethink the philosophy of API design.
BibTeX
@inproceedings{liu2024evaluation,
  author = {Bing Liu and Jianxiang Zhou and Dan Meng and Haonan Lu},
  title = {{An Evaluation Mechanism of LLM-based Agents on Manipulating APIs}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  url = {https://openreview.net/forum?id=dYOuwpy9TY},
}
Figure