Many multi-agent language-model systems are built around one "orchestrator" model. It splits a task into subtasks, assigns them to worker agents, reads their results, and decides what happens next. Recent measurements show that this one component causes about two thirds of all system failures. Its decisions also degrade as conversation history piles up in its context. Network engineers met this pattern a decade ago with software-defined networking (SDN), where a central controller computes forwarding rules and switches execute them. The centralized controller became a single point of failure. It was most fragile when control messages travelled over the same links as ordinary traffic, and it had no built-in way to recover from corrupted state. We propose to treat the LLM orchestrator as a control plane in exactly this sense and to import three SDN remedies.
The first remedy keeps orchestration decisions in a small, typed control table, separate from worker transcripts. The second lets a few replica orchestrators propose each change to that table, holding a short multi-agent debate only when the replicas disagree. The third periodically rebuilds the table from verifiable checkpoints so the system can recover from bad state. The claim we test has two parts. First, most orchestrator failures come from mixing control and data in one context, so separating them removes most of the problem. Second, debate pays for itself only on the minority of control decisions that are actually contested, and it can be added while keeping coordination cost under the budget where hierarchical orchestration stops being worth it.
Core Claim
Recognize Your Orchestrator models the orchestrator's decision entropy, meaning how uncertain it is about which agent to route to next. It finds the orchestrator accounts for 67.7% of failures versus about 32% for workers. It also finds that success collapses below 20% after roughly six workflow steps, as accumulated context swamps the model's focus. A related "reasoning trap" means heavy internal chain-of-thought makes things worse. That paper motivates our work: it locates the fault but proposes no mechanism to fix it beyond suppressing chain-of-thought.
We read its "dispersion" force as an in-band signalling problem. Canini et al. describe the analogous SDN deadlock, in which control packets share links with tenant traffic and so depend on the very data plane they configure. In an LLM orchestrator, worker outputs flow into the same context window that holds the plan. A verbose, wrong, or adversarial worker output therefore directly perturbs the next control decision.
Our core claim is that three changes should flatten the entropy-saturation curve, cut the orchestrator's failure share, and reduce how often a single faulty worker can hijack the plan, all at lower coordination cost than debating every step:
- putting the plan in an out-of-band control table,
- admitting worker outputs only as short typed status reports,
- gating changes to the table on agreement among replicas.
What is new relative to prior work is not "use debate to check the orchestrator." It is the control-plane decomposition, plus the specific hypothesis that debate belongs only on contested writes to control state.
Research Program
The control table is a compact structured record. It holds the current subtasks and their dependencies, assignments, status, and pointers to checkpoint evidence. Unlike The Orchestrator Bottleneck, whose ORCHESTRATE system extracts a dependency graph once and then chooses a sequential, parallel or hierarchical topology, our table permits cycles. Retries and validator–coder loops are explicit entries, which removes that paper's acyclic assumption.
Workers never write prose into the orchestrator's context. They return a status, a short claim, and a pointer to their full output, which stays in the "data plane" and is fetched only when a decision needs it. This is the LLM analogue of separate management channels in Iosifidis et al..
Every proposed change to the table, which we call a control write, is produced independently by $K$ lightweight orchestrator replicas (we plan $K=3$) that read the same table. Examples are marking a subtask done, re-decomposing, or rerouting work. We borrow the gating quantity from Epistemic Gain, Aleatoric Cost, which measures productive disagreement among debating agents as the Jensen–Shannon divergence of their beliefs. A write is committed directly if
$$\mathrm{JSD}(p_1,\dots,p_K) < \tau,$$
where $p_k$ is replica $k$'s distribution over candidate writes (estimated from a few samples) and $\tau$ is a threshold tuned on held-out workflows. Above $\tau$, the replicas run a short structured debate. We use the stance, refutation and concession rounds of FORD from Examining Inter-Consistency of LLM Collaboration, capped at two rounds and run only on the disputed write, followed by a majority commit.
FORD constrains the design in two ways. A stubborn weaker agent can drag a stronger one down, and a central judge added only about 0.6% accuracy. We therefore avoid a single judge and keep debates short. The replica-disagreement check is also our fault detector. It plays the role that comparing a live controller against its replica plays in Cyber-attack and Fault Detection using a Digital Twin.
Recovery borrows the idea, though not the proofs, of Renaissance, a self-stabilizing SDN control plane that returns to a legitimate state after arbitrary corruption. Every $M$ steps, or whenever replica disagreement persists, the table is re-derived from checkpoint evidence alone. Entries whose evidence fails verification are reopened.
We do not adopt the ledger-based controller authentication of IS2N or IOTASDN. We judge its overhead disproportionate for in-process agents.
All coordination tokens are metered against the roughly 15% overhead line reported by ORCHESTRATE.
Baselines
The naive reference is a single standard orchestrator that keeps the full transcript in context, as in the manager–executor setups studied by Zhu et al. The strongest existing orchestrator-side fix is Zhu et al.'s chain-of-thought-suppressed orchestrator (Qwen-Max "NoThink"). That paper reports a step success rate of 41.3%, versus 30.2% with thinking enabled, on its Inverse Workflow Generation environments. It is the strongest competitor because it attacks the same entropy failure with zero added cost. We will run it with the same base model as our replicas, and we will also run our system on top of it, because the two are complementary rather than exclusive.
On the cost side, the strongest system is ORCHESTRATE, which reports 0.58x execution cost relative to its sequential baseline on IncidentTriage. It also reports 28–42% aggregate savings at a 0.1–0.6 point quality cost, and a drop to 51% strategy-selection accuracy on ambiguous inputs. We run it with the same worker pool and token pricing.
We add an always-debate baseline in the style of ChainLM, which debates every reasoning step among personas before a central judge advances the chain. We apply it to every control write at an equal total token budget.
Three ablations isolate the ingredients:
- Table only: the out-of-band table with a single orchestrator.
- Table plus replica vote, no debate: majority vote of $K$ replicas with no debate step.
- Recovery removed: the full system without the periodic rebuild.
All methods share base models, worker pools, and tuning budgets, with $\tau$ and $M$ tuned on a split disjoint from test. Beating the field means two things on the IWG environments. First, step success rate above the NoThink figure with the same base model, with the gain persisting past step six where the baseline enters its chaotic regime. Second, coordination overhead below 15% and cost no worse than ORCHESTRATE on the enterprise suites.
Why not just…?
Why not just summarize or reset the orchestrator's context periodically? This is the most serious threat. If entropy saturation is purely a context-length effect, a free-text running summary might recover most of the gain. Our bet is that free-text summaries still carry worker prose, and with it worker errors, into the control context, whereas typed status reports do not. The table-only ablation compared against a periodic-summarization baseline at matched context length settles this. If summarization matches the table on clean workflows, we will narrow the contribution to robustness under faulty workers and to the replica-gated writes.
Why not just take a majority vote of orchestrator samples and skip debate? It is cheaper, and FORD's small judge gain suggests debate may add little over voting. The vote-only ablation exists for exactly this reason. Our prediction is narrower than "debate helps." Debate should beat voting only on high-disagreement writes, where the majority itself is often wrong, especially under ambiguous decompositions like those that drop ORCHESTRATE to 51%. If voting matches debate there as well, the protocol becomes vote-only and debate is dropped.
Why not just use a stronger or non-thinking orchestrator? Zhu et al. show that a better-configured orchestrator still fails in the chaotic regime. Scaling the model also does nothing about in-band hijacking by a misbehaving worker. Running our system on top of NoThink tests whether the gains add.
Why not remove the orchestrator and let peers coordinate? ORCHESTRATE reports that static parallel fan-out costs 1.38x on IncidentTriage because of redundant calls. Fully peer-to-peer coordination also inherits the synchronization and split-brain problems that decentralized SDN faces. We keep central control and replicate it narrowly.
Proposed Experiments
The primary testbed is the Inverse Workflow Generation environments from Zhu et al., which provide dense intermediate checkpoints. Those checkpoints serve both for evaluation and as the evidence our recovery step rebuilds from. We report step success rate by step index, end-task success, and the share of failures attributed to the orchestrator versus workers using that paper's attribution procedure. The decisive check is whether the step-success curve stays flat past step six and the orchestrator failure share falls well below 67.7%.
Cost and ambiguity are tested on IncidentTriage, DocProcess and ComplianceReview, plus the labelled decomposition set, from the ORCHESTRATE paper, if those are released. If not, we construct comparable workflows using the IWG procedure and say so. We report relative cost, task completion, coordination-token share, and accuracy on the ambiguous subset.
Robustness borrows the SDN threat model in which compromised edge devices manipulate a controller (Karmous et al.). In this setting, one worker returns confidently wrong or instruction-laden outputs. We measure hijack rate, defined as the fraction of runs in which a faulty worker's claim is committed to the plan. We found no reported result on Lacuna for orchestrator hijack rate under faulty workers, so all methods will be measured fresh.
Recovery is tested by corrupting the control table mid-run, deleting dependencies or flipping statuses, and measuring the number of steps until the table again matches checkpoint ground truth.
Two outcomes would falsify the central claim. One is that the table-only ablation gives no gain over transcript-based orchestration once context length is matched. The other is that gated debate never outperforms voting on high-disagreement writes. Either result would tell us where orchestrator failures actually come from.
References
- Recognize Your Orchestrator: An Entropy Dynamics Perspective for LLM Multi-Agent Systems
- Renaissance: A Self-Stabilizing Distributed SDN Control Plane
- The Orchestrator Bottleneck: Formal Coordination Strategies for Cost-Optimal Multi-Agent Enterprise Workflows
- SDN-Enabled Tactical Ad Hoc Networks: Extending Programmable Control to the Edge
- Epistemic Gain, Aleatoric Cost: Uncertainty Decomposition in Multi-Agent Debate for Math Reasoning
- Examining Inter-Consistency of Large Language Models Collaboration: An In-depth Analysis via Debate
- Cyber-attack and Fault Detection using a Digital Twin of the Controller Software
- Renaissance: A self-stabilizing distributed SDN control plane using in-band communications
- IS2N: Intent-Driven Security Software-Defined Network With Blockchain
- IOTASDN: IOTA 2.0 Smart Contracts for Securing Software-Defined Networking Ecosystem
- ChainLM: Empowering Large Language Models with Improved Chain-of-Thought Prompting
- Deep learning approaches for protecting IoT devices in smart homes from MitM attacks