Large Language Models (LLMs) have transitioned from passive text generators to active "agents" capable of calling external APIs to solve real-world problems. However, there is a fundamental difference between a model that can use a single tool and one that can orchestrate a complex workflow. While current models excel at isolated tasks, their performance often collapses when a user request requires a sequence of interdependent actions.
The Compositional Cliff
The primary challenge in modern agentic AI is not the capability of individual tools, but the logic required to chain them. Hiroki Furuta and colleagues demonstrated this "compositional cliff" in CompWoB, where even top-tier models like GPT-4 saw success rates plummet from 94% to 25% when individual web tasks were composed into sequences.
This failure occurs because sequential tool use is not just "more" work; it is a different kind of work. It requires the model to maintain a mental model of the environment's state and understand how the output of one action changes the prerequisites for the next.
Parameter Inheritance and Data Flow
In a simple tool-use scenario, a model might call a calculator to solve 2+2. In a sequential scenario, the model must perform what Pei Wang and the authors of MTU-Bench call parameter inheritance. This is the mechanism where the output of Tool A serves as the necessary input for Tool B.
Consider a request to "Email the weather forecast to John." The agent must:
- Search for "John" in a contact database to retrieve an email address.
- Call a weather API for a specific location.
- Pass both the retrieved email and the weather string into a mail-sending tool.
If the first tool returns a list of three "Johns," the agent must pause, reason about which one is correct, or ask a clarifying question—a capability Sung‐Hee Jung explores with DiaTool-DPO. As Pengfei He notes in TRAJECT-Bench, a critical bottleneck appears when agents move from short tasks to mid-length trajectories (3–5 tools), where the probability of a "broken link" in the data chain increases exponentially.
The Problem of State Management
For an agent to navigate a sequence, it must manage "state"—a record of what has been accomplished and what remains. Jiayang Song and Da Song highlight in StateEval that models frequently lose track of these dependencies.
This is particularly evident in "nested" sequences. Luis A. Lastras introduces the NESTFUL benchmark to specifically test these deep dependencies. When an API call is nested inside another, the model cannot simply execute and forget; it must hold the intermediate logic in its context window while preparing the next call. If the model fails to "look before it leaps"—a concept discussed by Anchun Gui in Look Before You Leap—it may trigger tools with irreversible side effects before it has all the necessary information.
Bridging the Gap with Process Supervision
To move beyond these limitations, researchers are shifting from evaluating final answers to evaluating the "trajectory" of the reasoning process. Sean M. Hendryx argues in ToolComp that process supervision—rewarding the model for each correct intermediate step in a tool chain—is more effective than outcome-based rewards.
By breaking down complex goals into a logical sequence of sub-tasks, as proposed in the UltraTool framework by Shijue Huang, agents can begin to handle the messy, multi-step workflows of the real world. The goal is to move from "tool-happy" models that call APIs at random to "agentic" models that plan, verify, and execute with the precision of a human expert.