Many clinical treatments are not a single pill at a single dose. Brain stimulation, for example, is set by frequency, intensity, target location, session count and timing after injury, and the best setting can flip as a patient recovers. Suppose several clinics each run their own adaptive search over these settings. Each clinic assigns patients to settings and learns which work. Sharing results would speed everyone up. But sharing is costly: it means data agreements, privacy review, harmonized reporting and staff time. It can also do harm. If one clinic mostly sees acute patients and another mostly sees chronic ones, pooling their results can cancel out real effects. We propose that each clinic, treated as an agent, sends a compact summary to a specific peer only when it predicts that the summary would change which setting that peer assigns next. We then test whether this "decision-relevant" sharing gets close to full-pooling performance at a small fraction of the messages, while avoiding the harm of pooling across mismatched sites.
The opportunity comes from two literatures that do not cite each other. Multi-agent learning has shown that communication should be priced and gated, and that broadcasting to everyone backfires. Clinical meta-research has shown that dose settings are high-dimensional and phase-dependent, and that pooling heterogeneous protocols destroys signal. Neither literature has studied coordinated, adaptive dose search where communication is both costly and possibly misleading.
Core Claim
Under protocol heterogeneity, the right unit of communication is not "my data" but "evidence that would move your next allocation." A sender can estimate this without querying the receiver. It keeps a mirror of the receiver's posterior (the receiver's current probabilistic belief about how well each setting works), built from the shared prior and every summary already exchanged between the two. Each site models its outcome as a Gaussian process, which is a smooth probabilistic regression over the dose-parameter vector $x$ (for example, frequency, intensity, target hemisphere and time since injury). Site $i$ sends its update to site $j$ only when
$$\mathrm{VoI}_{i\to j} = \mathbb{E}\big[\max_x \mu_j^{+}(x)\big] - \mu_j^{+}(\hat x_j) > c.$$
Here $\mu_j^{+}$ is the mirrored posterior mean at site $j$ after incorporating $i$'s summary, discounted by a learned cross-site similarity. The term $\hat x_j$ is $j$'s current best setting, evaluated under that updated posterior, and $c$ is the price of one message. Two things set this apart from existing gates. First, the cross-site similarity is part of the kernel and is estimated from the exchanged data. A clinic whose response surface differs from the receiver's will therefore have its value of information shrink toward zero automatically, instead of being pooled. Second, the gate needs no hand-set thresholds beyond the message price $c$, which has a direct meaning: cost per message in units of outcome.
Relation to Prior Work
Several papers supply the mechanism. The Utility of Communication in Coordinating Intelligent Agents introduced the idea that a message is worth its expected payoff gain to the recipient, and that messages which do not change the recipient's action should be suppressed. We reuse this principle in a statistical-learning setting with heterogeneous agents. Efficient Communication in Multi-Agent Reinforcement Learning via Variance Based Control gates requests and replies with thresholds $\delta_1, \delta_2$. It cut overhead 2–10× relative to SchedNet, but its thresholds are fragile across tasks. Our gate replaces both thresholds with a single interpretable price. A Principle of Targeted Intervention for Multi-Agent Reinforcement Learning showed that guiding one well-chosen agent beats broadcasting guidance to all agents. However, it fixes a single target and a static goal and assumes the influence structure is known. Our sender-to-specific-receiver gate chooses targets anew each round and learns who is similar to whom.
Two papers constrain the design. Learning with Limited Shared Information in Multi-agent Multi-armed Bandit proves that agents sharing less than logarithmically in the horizon cannot reach optimal regret. Its Balanced-ETC algorithm reaches group regret $O(N \log T + MN^2)$, but it assumes every agent faces identical, discrete arms. We take its lower bound as a floor our gate must respect. We relax its homogeneity assumption, which is exactly the assumption clinical data violate. Adaptive Discrete Communication Bottlenecks with Dynamic Vector Quantization for Heterogeneous Representational Coarseness found that adaptive bottlenecks gain more over fixed ones as heterogeneity grows (Reciprocal Rank 0.761 vs 0.701 on Boxing). That heterogeneity is perceptual, not a difference in reward between agents. It motivates our hypothesis that adaptive gating gains more as sites diverge.
On the clinical side, two papers motivate the problem. Comparative efficacy of different repetitive transcranial magnetic stimulation protocols for lower extremity motor function in stroke patients: a network meta-analysis pooled 38 trials. It found that low-frequency stimulation of the healthy hemisphere ranks first within a month of stroke (SUCRA 98.9%), while high-frequency stimulation of the damaged hemisphere ranks first afterward (SUCRA 95.8%). The analgesic effect of electroencephalographic neurofeedback for people with chronic pain: A systematic review and meta‐analysis showed that pooling across divergent frequency bands and electrode sites inflates heterogeneity until effects cannot be separated from protocol mismatch. Protocol for a bandit-based response adaptive trial ... the Vibe Up study already runs a Gaussian-process bandit for intervention allocation. It is centralized and uses four discrete arms; we decentralize it and price the messages. Credit-Budgeted ICPC-Style Coding documents the "Swarm Tax": agent teams with unconstrained communication spend their budget talking instead of solving. This is the failure mode our price $c$ is meant to prevent.
Research Program
We will build a semi-synthetic multi-site simulator whose response surfaces are anchored to the structure reported in the stroke network meta-analysis. Dose space covers frequency, hemisphere, intensity and session count. The optimum flips at one month post-injury. Sites differ in their mix of acute and chronic patients and in site-specific offsets. This is a stylized testbed, not a clinical model, and we will say so. Its value is that the ground-truth best protocol per phase is known, so regret is exact.
Each site runs a Gaussian-process upper-confidence-bound allocator, the same family Vibe Up uses. The kernel is a product of a dose kernel, a phase kernel and a site-similarity kernel. Messages are fixed-size summaries: inducing-point posterior statistics, so that every method sends the same number of bits per message. We will vary the number of sites (4 to 32), the degree of heterogeneity (from identical sites to fully opposed phase mixes) and the price $c$, and trace the frontier of regret against messages sent.
Two analyses accompany the experiments. The first asks when the mirrored posterior drifts from the receiver's true posterior, since the receiver also learns from its own patients between messages. The second checks empirically whether the gate shares at least logarithmically in $T$. If the gate falls below that floor, we add a minimum-rate safeguard, guided by the stingy lower bound in Balanced-ETC.
Baselines
The naive reference is fully independent sites: each runs its own Gaussian-process bandit and sends no messages. It costs nothing to communicate. Under strong heterogeneity it is also the baseline that pooling methods most often lose to, so beating it there is the minimum requirement.
The opposite reference is full broadcast with homogeneous pooling. Every site sends every round to every other site and treats all data as coming from one response surface. This is the "zero-cost communication" assumption and the computational analog of unstratified meta-analysis. We expect it to be strong when sites are identical and harmful at phase boundaries.
The strongest existing method under homogeneity is Balanced-ETC from Learning with Limited Shared Information in Multi-agent Multi-armed Bandit. We will run it on a discretized version of our dose grid, and on its own LSI-MAMAB simulator, using the same horizon and agent count. The paper gives only the $O(N \log T + MN^2)$ bound; no reported empirical regret value was found on Lacuna for Balanced-ETC. We will therefore run it ourselves and tune its balance level over the same grid budget that we give our price $c$.
The strongest heterogeneity-aware competitor is centralized full pooling with the same heterogeneous kernel: a Vibe Up–style central server that sees all data and models site and phase. It is an upper bound on outcome performance at maximal communication cost.
The ablation that isolates our key ingredient is a periodic-sync method. It uses the same heterogeneous kernel and the same message format, and its period is chosen so that its total message count exactly matches what our gate sent. A second ablation keeps our gate but removes the learned site-similarity term, which isolates the role of heterogeneity awareness.
We will count our method as winning if two conditions hold. At matched message counts, it reaches lower cumulative regret than periodic sync, by a margin outside seed-level confidence intervals over 30 seeds. It also reaches at least 90% of the centralized pooler's regret reduction over independent sites while using a stated fraction of its messages. All methods get identical simulator draws, identical patient arrival sequences and the same tuning grid size.
Why not just…?
Why not just pool everything centrally with a site-and-phase-aware model? If communication were truly free, this would be the right answer, and we include it as the upper bound. The proposal only matters in the regime where messages carry real cost: governance, privacy, harmonized reporting. The Swarm Tax in Credit-Budgeted ICPC-Style Coding shows the analogous cost overwhelming agent teams. The regret-versus-messages frontier will show how much of the centralized gain survives at each budget. If our gate cannot approach the centralized curve at substantially fewer messages, the method has no reason to exist.
Why not just sync on a fixed schedule? This is the real threat. Gaussian-process summaries are compact, and a well-chosen period may capture most of the value. Our ablation fixes the message budget to the gate's own count and compares only when messages are sent. We predict the gate concentrates messages in two places: early in the search, mirroring the communication spikes at episode start that VBC observed, and around the phase boundary. Periodic sync spreads messages evenly and wastes them in between. If matched-budget periodic sync performs as well as the gate, we will report that timing does not matter and that heterogeneity-aware kernels, not gating, carry the benefit.
Why not just cluster sites by covariates first, then pool within clusters? This works when the relevant covariate, such as time since injury, is known and recorded at every site. The PRECIS-2 review Applying the PRECIS-2 tool to pragmatic acupuncture trials found 84.9% of trials reported too little to evaluate delivery adherence, which suggests that the covariates needed for clustering are often missing. We will test pre-clustering in two simulator variants. In one, the phase covariate is observed and pre-clustering should be competitive. In the other, sites differ by unrecorded offsets, and only a similarity term learned from outcomes can detect the difference.
Decisive Checks
Three results would settle the proposal. First, with identical sites, the gate should match broadcast pooling and Balanced-ETC in regret. Failure here means the mirror or the gate is mis-specified. Second, under opposed phase mixes, homogeneous broadcast should do worse than independent sites at the minority-phase clinics. This would reproduce in simulation the washing-out effect from the stroke meta-analysis. The gate should beat both. Third, at matched message counts, the gate should beat periodic sync, with the gap widening as heterogeneity grows, the pattern DVQ reported for perceptual heterogeneity. If the third check fails, the contribution narrows to a heterogeneity-aware federated bandit, and we will present it as that.
References
- The Utility of Communication in Coordinating Intelligent Agents
- Efficient Communication in Multi-Agent Reinforcement Learning via Variance Based Control
- A Principle of Targeted Intervention for Multi-Agent Reinforcement Learning
- Learning with Limited Shared Information in Multi-agent Multi-armed Bandit
- Adaptive Discrete Communication Bottlenecks with Dynamic Vector Quantization for Heterogeneous Representational Coarseness
- Comparative efficacy of different repetitive transcranial magnetic stimulation protocols for lower extremity motor function in stroke patients: a network meta-analysis
- The analgesic effect of electroencephalographic neurofeedback for people with chronic pain: A systematic review and meta‐analysis
- Protocol for a bandit-based response adaptive trial to evaluate the effectiveness of brief self-guided digital interventions for reducing psychological distress in university students: the Vibe Up study
- Credit-Budgeted ICPC-Style Coding: When Agents Must Pay for Every Decision
- Applying the PRECIS-2 tool to pragmatic acupuncture trials: are they pragmatic? A systematic review