Learned network controllers are programs that tune congestion windows, bandwidth estimates or routes from live measurements. They are usually judged by running them in a simulator or emulator and then on a modest real testbed. The headline number pools thousands of calls, flows or traffic matrices. Those samples, however, come from only a handful of distinct real environments: a few carriers in one metro area, three unseen topologies, one intercontinental testbed run over twelve days. Clinical neuroimaging learned this lesson the hard way. When a cohort has 26 or 118 subjects, resampling within subjects inflates confidence, and models that look excellent collapse at a new scanner site. This proposal brings that discipline to network reinforcement learning. Network reinforcement learning means controllers trained by trial-and-error reward feedback. The proposal treats each real environment (a route, a carrier, a topology) as a "subject" and each testbed as a "site."
The idea fits in two sentences. First, we measure how much of a controller's real-world score comes from the environment it happens to run in, and how much from the controller itself, and we show how many distinct real environments are needed before a sim-to-real ranking can be trusted. Second, we test whether a small, heavily regularized "transfer predictor" can forecast which emulator results will survive deployment. The predictor is borrowed in spirit from physics-plus-learned-correction models in neuroimaging. We validate it with leave-one-environment-out testing, the way a biomarker must be validated on held-out sites.
Core Claim
The sim-to-real gap in network control is usually discussed as a training problem: train on real logs instead of emulators, as in the ACM MMSys 2024 Bandwidth Estimation challenge, or add uncertainty-gated fallbacks, as in RBWE. We claim it is equally an evaluation problem with a hidden small sample size. The statistically relevant count is the number of distinct environments, not calls or packets. If the controller-by-environment interaction is large, then reported improvements and rankings rest on far fewer effective samples than their sample counts suggest. The reported improvements in question include DeepCC's 175–300% queuing-delay reductions on LTE traces from three carriers (DeepCC), ENERO's generalization from three training topologies to three test topologies (ENERO), and the challenge leaderboard itself.
A simple variance model makes this precise. Let $y_{pec}$ be the score of controller $p$ in environment $e$ on session $c$, decomposed as $\mu + a_p + b_e + g_{pe} + \varepsilon_{pec}$. Here $a_p$ is the controller effect, $b_e$ the environment effect, $g_{pe}$ the controller-by-environment interaction with variance $\sigma^2_g$, and $\varepsilon$ session noise with variance $\sigma^2_\varepsilon$. With $E$ environments and $C$ sessions per environment, the variance of an estimated difference between two controllers is roughly
$$ \mathrm{Var}(\hat a_p - \hat a_{p'}) \approx \frac{2\sigma^2_g}{E} + \frac{2\sigma^2_\varepsilon}{E\,C}, $$
which follows from averaging independent interaction terms over environments and noise over all sessions. Adding sessions shrinks only the second term. When $\sigma^2_g$ is non-negligible, only adding environments helps. Pooled per-call confidence intervals implicitly set $\sigma^2_g = 0$.
We deliberately did not pursue the most obvious combination of these directions: training reinforcement-learning stimulation policies on simulated brain networks and "transferring" them to patients. No closed-loop stimulation data appear in the material that could test such a transfer. The open-loop SAINT cohort of 26 patients (SAINT DFC study) cannot falsify a learned policy. That project would therefore reproduce exactly the unvalidated sim-to-real optimism we want to measure. The neuroimaging literature contributes more here as a methodological lens than as an application domain.
How the Neuroimaging Literature Shapes the Design
Three neuroimaging results motivate the design. The Biopoint studies expand 118 children into roughly 1,600 bootstrapped graphs (Infomax ASD). The SAINT study reports a leave-one-out correlation of $R = 0.963$ on 26 patients without a sham arm. Both illustrate how within-subject resampling and tiny cohorts produce optimistic numbers. Network evaluations that bootstrap over calls from a few routes have the same structure.
AMP SCZ harmonized MRI across 34 sites and then decomposed variance into participant, site and platform components. Site variance stayed below 10% for structural and functional scans, but scanner platform still explained over 55% for diffusion MRI. We reuse this analysis directly: emulator versus real plays the role of scanner platform, and route or region plays the role of site.
Two papers supply the method. Physics-guided multi-view GNN corrects a Kuramoto oscillator simulation with a U-Net. The Kuramoto model is a mechanistic model of coupled oscillators; the U-Net is a learned image-to-image network. The correction raised agreement with real functional connectivity from $r=0.561$ to $0.748$. We ask whether the same "mechanistic simulator plus small learned residual" recipe can predict real network performance from emulator performance.
PR-GNN shows the second ingredient. A model with about 6,000 parameters and a loss that forces selected features to be shared across subjects outperforms larger unregularized models at $N=118$ (79.7% vs 75.3%). Our transfer predictor will be similarly lean. It will carry a penalty that favors features predictive of the gap consistently across environments, rather than features that fit one route.
Research Program
The first component is a variance audit. On the Microsoft Teams data from the MMSys 2024 challenge, we fit the mixed model above separately for real and emulated subsets, and jointly with a platform term. That data comprises 18,859 real calls from 6 production behavior policies and 9,405 emulated calls with programmed pathologies. The controllers are the six behavior policies. Their scores are the challenge's predicted audio and video opinion scores. Environments are defined by the finest grouping the release permits: country or route if available, otherwise day and endpoint clusters. If route-level labels are not released, the audit is coarser, and we will report it as a lower bound on environment variance. We repeat the audit on the DeepCC LTE traces, with carrier and mobility condition as environments. We also repeat it on TopologyZoo, rerunning ENERO-style routing agents across many of the 74 topologies rather than three.
The second component is a sample-size curve. By subsampling environments, we estimate the probability that the real-world ranking of controllers is recovered as a function of $E$ and $C$. From this we derive how many distinct environments a testbed needs. The deliverable is a concrete planning rule for future challenges and papers.
The third component is the transfer predictor. For each controller–environment pair, it takes cheap emulator-side quantities and predicts the real-minus-emulated score gap. These quantities include emulated score, sensitivity to emulated burst loss and policing, and action variability. The predictor is a sparse linear or small graph model with an environment-consistency penalty. It is trained and tested with strict leave-one-environment-out splits, never with split-by-call.
Baselines
The do-nothing reference treats emulator scores as real scores: rank controllers by emulated performance and report per-call bootstrap intervals. This is the default practice in ns-3 and emulator studies such as PBQ-QUIC. Our approach beats it if it recovers held-out real rankings more often, measured by Kendall's rank correlation, and if its intervals achieve nominal coverage on held-out environments.
The strongest existing evaluation practice is the challenge's own protocol: a live multi-day testbed over 600 intercontinental calls, which crowned Schaferct with 8.93/10 (MMSys 2024). We treat that protocol's design (calls, days, nodes) as a fixed budget. We then ask whether reallocating the same number of sessions across more environments, or adding our predictor, gives tighter and better-calibrated real-world estimates. Beating it means narrower environment-clustered intervals at equal session count with no loss of coverage. We will not claim to beat it if the variance audit shows small interaction variance.
For the strongest controller-level comparison, RBWE reports 17% lower overestimation than Schaferct and an 18.6% gain in 10th-percentile experience quality over Google Congestion Control in emulated replay (RBWE). We will re-estimate this gap with environment-clustered intervals and report whether it remains distinguishable from zero. No reported result was found on Lacuna for RBWE on the live global testbed.
For the transfer predictor, the baselines are the raw emulator score, and the emulator score with a single global linear calibration fitted on real data. All three receive the same real calibration environments and the same hyperparameter-search budget. Two ablations isolate the key ingredients. The first removes the environment-consistency penalty. The second replaces leave-one-environment-out splits with split-by-call, which we expect to show inflated accuracy, as in the Biopoint and SAINT cautions.
Why not just…?
Why not just run more real calls? More calls shrink only the session-noise term. If $\sigma^2_g$ is substantial, ten times more calls on the same routes leaves ranking uncertainty nearly unchanged. The sample-size curve tests this directly. If it shows that the existing session counts already saturate ranking stability, the premise is wrong, and we will report that the current testbeds are adequate.
Why not just bootstrap over calls, as everyone does? Call-level bootstrap assumes calls are independent, which is the same mistake as bootstrapping Biopoint subjects into 1,600 graphs. We measure the actual coverage of per-call intervals on held-out environments. If coverage is near nominal, this alternative is fine, and the proposal's evaluation contribution shrinks to a confirmation.
Why not just train on real logs and skip the emulator question? That is what the MMSys challenge and RBWE do, and it addresses training. It does not tell anyone how many environments their evaluation needs. Offline-trained policies also still require held-out real evaluation, whose reliability is exactly what we quantify. This alternative is complementary, not competing.
Why not just calibrate the emulator to real traces, digital-twin style? This is the real threat to the transfer predictor. If a global calibration of emulator output closes most of the gap, the learned residual adds nothing. We include global calibration as a baseline at equal data. The predictor counts as a contribution only if it beats it on held-out environments. The Kuramoto result ($r$ 0.561 to 0.748) suggests a learned residual can help, but that evidence comes from another domain and decides nothing here.
Proposed Experiments and Decisive Checks
The decisive check is the variance audit. If the controller-by-environment component is below about 10% of total variance on the Teams data and the LTE traces, then the existing small-environment evaluations are statistically sound. The 10% threshold echoes the level AMP SCZ treated as acceptable site variance. In that case the paper becomes a reassuring negative result with a sample-size rule. If the component is large, we expect a large emulator-versus-real platform term, consistent with the challenge's observation that emulator-trained models struggled live. We then report which published comparisons lose significance under environment-clustered intervals.
On TopologyZoo, we compare ENERO-style generalization estimated from three test topologies with estimates from dozens of held-out topologies. Specifically, we measure maximum link utilization relative to the local-search optimum, giving a direct picture of how far a three-topology estimate can sit from the population value.
For the transfer predictor, success is two-part. It must beat global calibration on held-out-environment rank correlation and calibration error on the Teams data. Its advantage must also survive the leave-one-environment-out split rather than appearing only under split-by-call.
References
- ACM MMSys 2024 Bandwidth Estimation in Real Time Communications Challenge
- Robust Bandwidth Estimation for Real-Time Communication with Offline Reinforcement Learning
- Wanna Make Your TCP Scheme Great for Cellular Networks? Let Machines Do It for You!
- ENERO: Efficient real-time WAN routing optimization with Deep Reinforcement Learning
- A Study on dynamic functional connectivity and efficacy prediction of SAINT for rapid antidepressant effects and relief of suicidal ideation
- Graph embedding using Infomax for ASD classification and brain functional difference detection
- The MR neuroimaging protocol for the Accelerating Medicines Partnership® Schizophrenia Program
- Physics-Guided Multi-View Graph Neural Network for Schizophrenia Classification via Structural-Functional Coupling
- Pooling Regularized Graph Neural Network for fMRI Biomarker Analysis
- PBQ-Enhanced QUIC: QUIC with Deep Reinforcement Learning Congestion Control Mechanism