Systems that split a task across several language-model agents have to decide, many times per task, whether to send the next subtask to an agent role they already have or to create a new specialist. A role here means a system prompt plus a small set of tools. Today that decision is made in one of two ways. A human fixes the roles in advance, or a planning model invents them on the spot. Neither approach states what a new role costs or what it buys. The result is a familiar pattern. Some systems carry too few roles and push errors downstream. Others carry too many, pay several times the tokens of a single agent, and confuse themselves with oversized tool menus.
We propose to make this an explicit accounting decision, borrowed from nonparametric Bayesian clustering. That field groups data without fixing the number of groups in advance: a point opens a new group only when it is far enough from every existing one. In its simplest deterministic form, opening a group costs a fixed penalty, and every point pays the distance to its group. We set that penalty to the measured token overhead of instantiating a role. We set the distance to a calibrated estimate of how much more likely a subtask is to fail when handled by an ill-fitting role. Under that substitution, the clustering rule becomes a cost-minimizing policy for growing a role library online. The question is whether this policy yields better accuracy per token than letting a language model hire agents freely.
Core Claim
Our claim is that a penalized online clustering objective is a better governor of role creation than either free-form language-model generation or a fixed number of clusters. The objective is in the style of small-variance asymptotics, the limit in which a Dirichlet process mixture collapses into a deterministic "assign or open new cluster" rule. It works better when its penalty is tied to measured cost rather than tuned. It matters most when the mix of tasks shifts over time.
Concretely, we minimize the following expected cost over the number of roles $K$ and the assignment $z_i$ of each subtask $i$ to a role:
$$ J = \lambda K + c \sum_i f\big(d(x_i, \mu_{z_i})\big). $$
The symbols are defined as follows. $\lambda$ is the average token cost of standing up one role, meaning its prompt, its tool documentation, and its amortized share of context. $x_i$ is the subtask's representation. $\mu_k$ is role $k$'s centroid. $d$ is a distance. $f$ is a calibration curve mapping distance to the increase in failure probability. $c$ is the token cost of one failure, including retries and the downstream damage it causes.
This form directly descends from the objective in DP-space. That work derived a loss of $\lambda K$ plus a dimension penalty plus squared distances, and proved that its alternating updates decrease the loss monotonically. Our change is to replace a geometric threshold that has to be tuned with one whose units are tokens and can be measured. This responds directly to the known weakness of nonparametric methods: they swap the choice of $K$ for a hyperparameter that still needs tuning.
The claim is falsifiable in three ways. First, with $\lambda$ set from measurements, the policy should sit near the knee of the accuracy-versus-tokens curve traced by sweeping $\lambda$. Second, the number of roles should grow when a new kind of task enters the stream and plateau within a stationary stream. Third, on a stream whose task mix shifts, the policy should beat $k$-means even when $k$-means is given the best fixed $K$ chosen in hindsight.
Research Program
The work has three parts. The first is a modular role layer. The second is the online clustering policy with calibration. The third is a split-and-retire step driven by execution outcomes.
The role layer. The modular-design literature supplies the main constraint. SMCTC and FaceX-Zoo both show that separating the execution loop from swappable strategy components lets new strategies be compared without touching the loop. We apply that separation here. The role library is a plug-in behind a fixed interface. The interface takes a subtask description and returns a role prompt and a tool subset. The executor above it stays unchanged: either the dynamic decomposer of TDAG or the graph-building planner of TSAS. Any difference in results is therefore attributable to how roles are formed, not to orchestration.
The clustering policy and calibration. Each subtask is represented at routing time by a sentence embedding of its description. After execution, the role's centroid is updated with an extended record that also includes the tools actually called and the outcome. As a result, roles drift toward what their members really needed, not only what they sounded like.
The calibration curve $f$ is fitted on a pilot set. We deliberately route subtasks to roles at varying distances and record their failure rates. This step is what separates the proposal from simply running DP-means on embeddings. The distance is converted into expected failures, so $\lambda$ and the distance term share units.
Tool subsets per role are the union of tools used by successful members, capped and pruned by usage. This gives a principled version of the "role firewall" against tool overload that Test-Time Tool Evolution motivates. That work found accuracy falls when orchestrators face too many tools, and it pruned tools only by heuristic similarity and usage counts.
Split and retire. Periodically, a role whose members fail at a high rate is tested for a split. The split is accepted if two sub-centroids lower $J$ by more than $\lambda$. Roles whose membership falls to zero over a window are retired. Both moves follow from the same objective, so no separate rule is introduced.
We will also test the subspace variant from DP-space, in which each role is a low-dimensional subspace with a per-dimension penalty. This is an ablation, not the default. It is justified only if roles really differ in how many independent requirements they span.
Baselines
The naive reference point is a single agent with access to the full tool registry, run on the same backbone model. Single-agent or Multi-agent Systems? Why Not Both? reports that multi-agent systems use 4x to 220x more prefill tokens and can lose to a single agent on easy tasks. Any role policy must therefore beat this baseline on accuracy at a token cost we report, not just on accuracy alone. We also include that paper's cascade, which tries a single agent first and escalates on failure. The bar is to match its accuracy at fewer tokens on easy-task segments.
The strongest existing competitors are the two language-model-driven hiring schemes, and each is run with our plug-in disabled.
TDAG creates sub-agents dynamically and retrieves cached skills by SentenceBERT similarity. It reports an average fine-grained score of 49.08 on ItineraryBench and a 73.5% success rate on TextCraft. Its cascading-task-failure rate on ItineraryBench is 4.35%, against 34.78% for plan-and-execute.
TSAS has a planning agent synthesize a team and a static execution graph. It reports 47.18% accuracy on the GAIA test set at $31.58 per run. Its distilled GraphGen-14B planner reaches 57.57% on GAIA validation.
For scientific tasks, TTE-Zero reports 0.62 accuracy on SciEvo against 0.56 for the best baseline it compared to.
We will rerun every one of these with the same backbone, tool registry, and token budget. Our tuning budget for $\lambda$ and calibration comes from a held-out pilot split equal in size to any prompt-tuning budget we give the baselines. The published numbers are reference points, not targets, because backbones differ. Beating a competitor means equal or higher task accuracy at significantly fewer total tokens, or higher accuracy at equal tokens. Significance is assessed by paired bootstrap over tasks.
Two ablations isolate the key ingredients. The first is $k$-means on the same features, with $K$ chosen by oracle on the full stream. This tests whether nonparametric growth matters at all. The second is DP-means with $\lambda$ tuned by grid search instead of set from measured cost. This tests whether the cost-grounded penalty matters. A third ablation removes outcome-based centroid updates, which tests whether execution feedback matters beyond description similarity.
Why not just…?
Why not just let the planning model decide when to hire a new agent, as TDAG and TSAS already do? This is the incumbent, and on accuracy it may be hard to beat. Our bet is on cost and stability, not raw accuracy. Free-form hiring has no term that charges for an extra role. Separately, TSAS reports that iterative critique loops to refine workflows raised token cost 50 to 70 times for negligible gains, which suggests that more language-model deliberation is not a cheap fix. The head-to-head on matched backbones settles it. If TDAG or TSAS reach the same accuracy at the same tokens, the proposal adds only interpretability, and we will say so.
Why not just run $k$-means or a density-based clusterer on subtask embeddings? This is the most serious threat. On a stationary benchmark, a well-chosen $K$ may do as well as anything. The nonparametric rule should earn its place only in two settings: when task heterogeneity shifts mid-stream, and when no labeled hindsight is available to choose $K$. For that reason the decisive comparison uses a shifting stream and gives $k$-means its oracle $K$. If oracle $k$-means still matches us there, the contribution shrinks to the cost-grounded threshold. A density clusterer with a minimum-cluster-size knob has the same unpriced-hyperparameter problem, and it gets the same grid budget.
Why not just route easy tasks to a single agent and hard ones to a fixed team? Gao et al. report up to 88.1% cost savings from this kind of routing. That makes it a strong token baseline, and it is included as one. It decides only whether to use a team. It does not decide which specialists the team should contain, or how many tools each should see. Those are the questions our policy answers. The two methods compose, and we will also test our layer behind their router.
Proposed Experiments
Sanity check. We first confirm that our implementation of the small-variance objective is correct by reproducing DP-space on Hopkins 155, where the paper reports roughly 3.3% segmentation error in about 2 seconds.
Main experiment. The main experiment is a single shuffled stream that enters domains in blocks. It draws on the GAIA validation set (GAIA), ItineraryBench, TextCraft, and SciEvo. The block structure means the true heterogeneity changes at known points. We report task accuracy, total and per-task prefill and decoding tokens, cascading-task-failure rate where defined, role count over time, and mean tools exposed per role.
Decisive checks. Four results decide the outcome. First, the $\lambda$ sweep: the measured $\lambda$ should land within the flat region near the knee of the accuracy-versus-tokens curve. If it falls far from the knee, the cost-grounding argument fails. Second, the role count should jump at block boundaries and plateau within blocks. Third, on the shifting stream, the policy should beat oracle-$K$ $k$-means on accuracy per token. Fourth, accuracy against tools-per-role should reproduce the overload drop that TTE describes for the flat-registry baseline, and should not reproduce it for our roles.
If the first three checks hold, multi-agent systems gain a defensible, tunable-by-measurement rule for when to hire. If they fail, we will have shown cleanly that free-form hiring or fixed $K$ is already good enough.
References
- DP-space: Bayesian Nonparametric Subspace Clustering with Small-variance Asymptotics
- SMCTC: Sequential Monte Carlo in C
- FaceX-Zoo: A PyTorch Toolbox for Face Recognition
- TDAG: A multi-agent framework based on dynamic Task Decomposition and Agent Generation
- On-Demand Multi-Agent Workflow Synthesis for Multimodal Long-Horizon Reasoning
- Beyond Static Tools: Test-Time Tool Evolution for Scientific Reasoning
- Single-agent or Multi-agent Systems? Why Not Both?
- GAIA: a benchmark for General AI Assistants