Abstract
Decomposing hard problems into subproblems often makes them easier and more efficient to solve. With large language models (LLMs) crossing critical reliability thresholds for a growing slate of capabilities, there is an increasing effort to decompose systems into sets of LLM-based agents, each of whom can be delegated sub-tasks. However, this decomposition (even when automated) is often intuitive, e.g., based on how a human might assign roles to members of a human team. How close are these role decompositions to optimal? This position paper argues that asymptotic analysis with LLM primitives is needed to reason about the efficiency of such decomposed systems, and that insights from such analysis will unlock opportunities for scaling them. By treating the LLM forward pass as the atomic unit of computational cost, one can separate out the (often opaque) inner workings of a particular LLM from the inherent efficiency of how a set of LLMs are orchestrated to solve hard problems. In other words, if we want to scale the deployment of LLMs to the limit, instead of anthropomorphizing LLMs, asymptotic analysis with LLM primitives should be used to reason about and develop more powerful decompositions of large problems into LLM agents.
Introduction
The field of LLM-based agent design is currently in a "Wild West" phase. As these systems scale from simple chatbots to complex organizational workflows, developers often rely on "intuitive" or "anthropomorphic" designs. They build multi-agent systems by mimicking human structures—assigning roles like Manager, Developer, or QA—without a rigorous understanding of whether these structures are computationally optimal.
This position paper argues that while such designs might suffice for small-scale demonstrations, they risk becoming significant bottlenecks as problem complexity grows. To address this, the authors propose a formal framework for Asymptotic Analysis with LLM Primitives (AALPs). By treating a single LLM forward pass as the atomic unit of cost, we can mathematically prove which agent architectures are fundamentally more efficient as problems scale.
The AALP Framework
The core of AALP is the definition of the "LLM Primitive": a single execution of an LLM. By abstracting away the internal "black box" mechanics of the model, the framework focuses on the efficiency of the orchestration layer.
Key components include:
- The LLM Primitive: The atomic unit of computation is defined as a single LLM execution.
- Cost Modeling: The cost of an LLM call is assumed to be proportional to the model size ($m$) and the number of input tokens ($n$).
- Capabilities Sets: Models are defined by their reliable capabilities, distinguishing between "Generalists" (large $m$, broad capabilities) and "Specialists" (small $m$, narrow capabilities).
- Oracle Decomposition: This theoretical tool explores how breaking a problem into sub-problems (e.g., splitting a file into functions) affects overall scaling laws.
Using these tools, the authors perform "asymptotic separations"—mathematical proofs showing that one architecture is fundamentally superior to another as problem size approaches infinity.
Case Studies in Agent Orchestration
To demonstrate the power of AALP, the paper analyzes three theoretical problems that represent common agent workflows. These examples illustrate how moving from a "naive" or "optimistic" approach to an "optimized" decomposition yields significant efficiency gains.
- k-Task Routing: Choosing between $k$ different specialized tasks. The analysis shows that using a "Delegator" model to route tasks to many small "Specialist" models provides a linear speedup ($\Theta(k)$) compared to a single giant "Generalist."
- Iterative Code Debugging: Fixing $b$ bugs in a codebase of $n$ tokens. A "Naive" approach that feeds the entire history back into the model leads to quadratic cost blowup. Conversely, a "Focused" approach where agents target specific code chunks provides a $\Theta(bk^2)$ improvement.
- Evolutionary Optimization: Using LLMs to mutate and improve solutions. By breaking a global problem into local "blocks" (Local Mutation), the expected cost drops from exponential ($\Theta(2^k)$) to polynomial ($\Theta(k)$).
Key Results and Insights
The theoretical separations derived in the paper suggest that for large-scale systems, the orchestration strategy is often more critical than the raw power of the underlying LLM. The table below summarizes the theoretical improvements found:
| Problem | Optimized Approach | Theoretical Improvement |
|---|---|---|
| k-Task Routing | Specialist models + Delegator | $\Theta(k)$ (Linear) |
| Code Debugging | Focused agents on sub-functions | $\Theta(bk^2)$ (Quadratic) |
| Evolution | Local mutation on sub-blocks | $\Theta(2^k / k)$ (Exponential speedup) |
A central takeaway is the need to stop anthropomorphizing agents. Designing agents to "act like humans" is often inefficient. Instead, they should be treated as "computational objects" optimized for algorithmic efficiency. Specialization and decomposition are not just design choices; they are fundamental scaling laws.
Limitations and Future Directions
While AALP provides a powerful lens, several challenges remain:
- Reliability vs. Stochasticity: The current analysis assumes 100% reliability within a capability set. Real-world LLMs are stochastic and prone to error, necessitating the integration of error-correction costs into future formulas.
- The "Oracle" Problem: The framework often assumes an "Oracle" for decomposition. Research into how agents can automatically and optimally decompose problems is a critical next step.
- Beyond Text: The current focus is on token-based costs. Extending this to multimodal agents (vision, action) will require new primitive definitions.
- Asynchrony: Most examples provided are sequential. Analyzing parallel and asynchronous workflows represents a major open research direction.
By moving toward a rigorous complexity theory for agents, the community can move beyond intuition and unlock the true potential of large-scale LLM orchestration.
Summary updated