TL;DR
This paper introduces a novel approach to enable tool-using AI agents to learn API functionalities solely from in-context demonstrations, bypassing the need for traditional, often unavailable or outdated, API documentation. They demonstrate that this "documentation-free" learning is challenging but achievable, and that self-exploration combined with LLM-based evaluation can significantly improve agent performance.
What Problem Does This Paper Address?
Modern AI agents are increasingly empowered with "tools" – external Application Programming Interfaces (APIs) – to perform complex tasks. Traditionally, these agents rely on API documentation to understand how to use these tools, including their purpose, input parameters, and return values. However, this documentation is frequently missing, outdated, or inconsistent, creating a significant bottleneck for developing reliable and general-purpose AI agents. This paper tackles the problem of learning API functionality without any prior documentation, forcing agents to infer how to use tools solely from examples of their usage.
Methods
The core of the paper's methodology revolves around learning API functionality from "in-context demonstrations." These demonstrations are examples of API calls, often paired with the task they are intended to solve and the preceding steps. The authors propose several methods for processing these demonstrations to inform an agent:
- Direct Expert Demonstrations (DxD): The agent is directly provided with a set of sampled demonstrations for each API.
- Generated Documentation (GD): An LLM is used to generate documentation from the demonstrations.
- Generated Documents with Example Calls (GDEC): Combines generated documentation with actual function call examples.
- Oracle Baseline (OD): Uses the original, ground-truth documentation (for comparison).
A key innovation is the self-exploration phase. After initial learning from expert demonstrations, the agent is allowed to "play" with the APIs on training tasks. The experiences (function calls and their outcomes) are then evaluated by an LLM-based "judge" to provide richer feedback than just task success. This feedback is used to update the agent's understanding of the API functionality through methods like:
- Direct Experience (DE): Directly using the evaluated experiences.
- Updated Documentation (UD): Updating the generated documentation with experiences.
- Regenerated Documentation (RD): Regenerating documentation from scratch using both expert demos and experiences.
- Attached Guidelines (AG): Summarizing experiences into guidelines and attaching them to the generated documentation.
Datasets
The authors evaluate their methods on three API benchmarks designed for multi-step queries:
- WorkBench (Styles et al., 2024): Uses pre-computed trajectories from a GPT-4 agent.
- T-Bench (Yao et al., 2024): Uses pre-computed trajectories from a GPT-4.5 agent.
- CRMArena (Huang et al., 2025): Uses pre-computed trajectories from a GPT-4o agent.
These benchmarks provide sandbox environments for executing API calls and include tool execution results.
Key Insights and Ideas
- Documentation is a major bottleneck: The paper strongly reinforces that the lack of reliable API documentation is a significant hurdle for building capable tool-using agents.
- Parameter filling is critical and difficult: A recurring problem across all methods is accurately understanding and filling API parameters. This is highlighted as a central challenge, leading to substantial drops in success rates when handled incorrectly.
- Self-exploration and LLM feedback are powerful: The self-exploration phase, coupled with LLM-based evaluation, proves to be a highly effective mechanism for agents to learn from their mistakes and improve their understanding of API usage.
- Guidelines are more effective than raw documentation: Attaching generated guidelines (AG) based on self-exploration experiences significantly boosted performance compared to just using generated documentation (GD).
- Misleading error messages are problematic: The paper points out how poorly designed error messages from APIs can mislead both the agent and the evaluation system, hindering learning.
Key Results
- Learning API functionality from demonstrations alone is a "non-trivial challenge" even for state-of-the-art LLMs.
- Providing explicit function calls and natural language critiques (via self-exploration and LLM judges) significantly improves parameter filling and task success.
- The Attached Guidelines (AG) method showed a notable 12% improvement over Generated Documentation (GD) on the WorkBench dataset.
- In T-Bench, incorrect parameter formatting (specifically for
order_id) led to misleading error messages and significantly degraded performance, with the AG method's success rate increasing from 10% to 49% when specific guidelines were hardcoded. - Open-source models like Qwen3, when using the DxD method, outperformed even some OpenAI models on WorkBench, suggesting the potential of these models in documentation-free scenarios.
Limitations and Future Work
- Suboptimal Expert Demonstrations: The heuristic used to extract expert demonstrations (only using successful trajectories) can lead to suboptimal examples. The authors suggest using LLM-based evaluators to signal correct steps, even within suboptimal trajectories.
- Iterative Generation: Documents were generated for each function independently. Future work could explore group learning for functions that share parameters.
- Complex Scenarios: The current work focuses on a single set of APIs. Extending this to scenarios with multiple data sources and API sets (Model-Context Protocol) is a promising direction.
- Human-in-the-Loop: Incorporating human feedback to extract online demonstrations could pave the way for imitation learning algorithms.
- Filtering Suboptimal Demonstrations: Explicitly mitigating or filtering out suboptimal demonstrations is an area for future research.
Summary updated