The Coordination Tax and the Heterogeneity Dividend: Break-Even Conditions for Multi-Agent LLM Pipelines

Andreas Zettl, Diego GosmararXiv 2026 · arXiv

TL;DR

Multi-agent LLM pipelines are often justified by their supposed "reasoning" capabilities, but they come with a hidden "coordination tax." This paper provides a formal economic model to show that for many coding tasks, complex multi-agent architectures are significantly slower and more expensive than a single strong model call, often failing to provide any accuracy boost.

What Problem Does This Paper Address?

The AI community is currently enamored with multi-agent systems—pipelines where different "agents" take on roles like Planner, Coder, and Reviewer. While these are praised for their flexibility and modularity, their actual runtime costs are rarely measured rigorously.

The authors identify a fundamental tension:

  1. The Coordination Tax: The overhead incurred by breaking a task into multiple steps (generating more tokens) and the latency added by interoperability protocols (like the Open Floor Protocol).
  2. The Heterogeneity Dividend: The potential savings gained by routing simpler sub-tasks to cheaper, faster models.

The paper asks: Under what conditions does the dividend actually pay for the tax?

Methods

The authors develop a closed-form analytical model to define the "break-even" point for multi-agent pipelines. They introduce three key variables:

  • $\alpha$ (Decomposition Inflation): How many more tokens the pipeline generates compared to a single call.
  • $r$ (Price Ratio): The cost of the cheap model relative to the strong model.
  • $f$ (Routable Fraction): The percentage of work that can be safely offloaded to the cheap model.

They derive a "Feasibility Bound": a pipeline can never be cheaper than a single strong call if $\alpha \ge 1/r$. If your decomposition generates 40x more tokens to use a model that is 40x cheaper, you are at best breaking even.

To test this, they ran 2,430 experimental runs across six different architectural conditions, ranging from single model calls to complex three-stage pipelines using the Open Floor Protocol (OFP).

The six experimental conditions tested in the study, comparing single calls, routers, and multi-agent pipelines.

Datasets

The study evaluated performance on 135 coding tasks across two difficulty tiers:

  • HumanEval: 50 function-level tasks (simpler).
  • ClassEval: 85 class-level tasks (more complex, requiring multiple interacting methods).

They used Claude Opus 5.5 as the "strong" model and Claude Haiku 5.5 as the "cheap" model.

Key Insights and Ideas

  • The Protocol is Not the Problem: One of the most surprising findings is that the interoperability protocol (OFP) is incredibly efficient. It added only 7.2 ms per handoff, accounting for just 0.11% of total wall time. The "tax" is almost entirely paid in tokens, not protocol overhead.
  • The "Reasoning" Trap: Cheap models often "reason" more verbosely than strong models. In the mixed pipeline, the cheap model (Haiku) inflated token volume so much that it wiped out a significant portion of its price advantage.
  • Task Routing > Role Routing: The authors found that it is much more efficient to use a router to decide which model should handle the entire task rather than splitting every task into roles.
  • Decomposition Often Fails to Scale: For self-contained coding tasks, adding a separate planner and reviewer didn't add new information; it just added cost. In fact, the multi-agent pipelines were often less accurate because errors could propagate through the chain.
The break-even threshold as a function of decomposition inflation. Above the $f^*=1$ line, no routing policy can make the pipeline cheaper than a single call.

Key Results

  • Cost & Latency: A standard planner-coder-reviewer pipeline (using only the strong model) cost 3.22x as much and took 3.24x as long as a single call, with zero gain in accuracy.
  • The Mixed Model Failure: Moving the planner and reviewer roles to the cheap model (Haiku) reduced costs to 0.39x of the uniform pipeline, but it was still 1.27x dearer than a single strong call and 8.1 points less accurate.
  • The Accuracy-Cost Frontier: As shown in Figure 3, single-call models and simple routers consistently dominate multi-agent pipelines. The pipelines (filled squares) sit far to the right (more expensive) and often lower (less accurate) than the single-call baselines.
The accuracy-cost frontier shows that single calls and routers (open circles) generally outperform multi-agent pipelines (filled squares) on both cost and accuracy.

Limitations and Future Work

  • Task Complexity: The tasks used were self-contained. The authors acknowledge that for "under-specified" or "long-horizon" tasks, a multi-agent approach might finally show its worth.
  • Vendor Monoculture: Both models were from the same vendor (Anthropic). Cross-vendor pipelines might behave differently.
  • Unmeasured Benefits: The study did not measure the "soft" benefits of protocols, such as easier debugging, audit trails, or the ability to swap out agents from different organizations dynamically.
  • Future Directions: The authors suggest testing "fan-out" architectures (parallel sub-tasks), which is the only regime where multi-agent systems can theoretically beat a single call on wall-time latency.

Summary updated

Abstract

Multi-agent LLM pipelines built on interoperability protocols such as the Open Floor Protocol (OFP) or A2A are usually justified on capability grounds, with their runtime cost treated as an overhead of unknown size. Comparing such a pipeline with one model call is underspecified, because a pipeline can give each role a different model. We give a closed-form model that sets the coordination tax (decomposition overhead plus per-handoff protocol cost) against the heterogeneity dividend from routing work to a cheaper model, with a break-even threshold and a feasibility bound, $α< 1/r$, and measure every term, with OFP as the protocol under test, on 135 HumanEval and ClassEval tasks in 2430 runs of six architectures with Claude Opus 5.5 and Haiku 5.5. The protocol adds 7.2 ms per handoff, 0.11% of pipeline wall time. Decomposition does not pay on these tasks: a planner-coder-reviewer pipeline costs 3.22x as much as a single strong call, takes 3.24x as long, and solves no more tasks. Moving planner and reviewer to the cheap model cuts the pipeline's cost to 0.39x but changes pass@1 by -6.7 points, leaving it dearer, slower and less accurate than the single call, and the break-even forecast misses by -19% because the cheap roles inflate token volume. The saving lies in choosing a model per task instead: an oracle choice between the two single calls solves 94.3% of runs at half the strong call's cost per solved task.
BibTeX
@misc{zettl2026coordination,
  author = {Andreas Zettl and Diego Gosmar},
  title = {{The Coordination Tax and the Heterogeneity Dividend: Break-Even Conditions for Multi-Agent LLM Pipelines}},
  year = {2026},
  eprint = {2610.37700},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.37700},
}
Figure