Controllers for traffic lights, network queues and similar systems are usually trained in simulators, and the simulators are usually tested for robustness with perturbations that are fixed for a whole run. A typical test lowers braking capability for the entire episode or adds constant sensor noise. That is a hidden version of the static loading assumption engineers use when they drop time derivatives and study only equilibria. Structural mechanics already knows when that shortcut is safe. A quasi-static test is valid only when loads change slowly compared with how fast the system settles, and it fails badly under fast impacts, as in Experimental and Numerical Investigation of 3D Printing PLA Origami Tubes under Quasi-Static Uniaxial Compression. Likewise, a static-equilibrium design transferred to a real 3D-printed mechanism only because it was actuated slowly (Neuromechanical Autoencoders). We propose that the same rule governs the gap between simulation and deployment for queueing controllers. A disruption of a given size hurts much more when it arrives faster than the network's queues can drain, and current benchmarks, which apply shifts as constants, largely cannot see this.
The proposal has two parts. First, we extend the traffic-signal benchmark Sim2Signal with transient perturbations. Each one has the same time-averaged size as an existing constant perturbation but a controlled onset speed, and the transient versions can be combined, for example rain arriving together with a detector dropout and a demand surge. Second, we test whether a controller that carries an explicit, online-recalibrated estimate of queue-length distributions and spillback risk recovers the performance lost under fast onsets. Spillback means a queue growing backward far enough to block the intersection upstream. The estimate comes from a differentiable probabilistic queue model (Osorio, Flötteröd and Bierlaire). The comparisons are against randomized training and against a generic recurrent policy.
Core Claim
Let $\tau$ be the onset time of a disruption, meaning how long a drop in capacity or a rise in demand takes to go from zero to full size. Let $T_r$ be a link's relaxation time, meaning how long its queue takes to return to typical length after a step change at nominal service rate. We claim that the transfer loss of a controller trained under stationary conditions depends mainly on the ratio $\tau / T_r$ rather than on the size of the shift. The loss should be small and flat when $\tau \gg T_r$, which is the quasi-static regime. It should rise sharply once $\tau \lesssim T_r$, because queues then build faster than a controller tuned for equilibrium conditions can respond. This form of the claim comes from the quasi-static analogy: inertia-like effects, here queue growth that has not yet been served, matter only when loading outpaces relaxation.
The second half of the claim concerns the remedy. Sim2Signal reports that explicit estimation of the disruption beats randomization aimed at making the policy insensitive to it. We claim this advantage is largest in the fast-onset regime, and only when the estimator models transient queue dynamics. An estimator built on a steady-state queue formula should behave like the naive policy there.
Positioning Against Prior Work
Two strands motivate this work. Sim2Signal breaks the reality gap for signal control into observation, action, transition and reward components across 10 calibrated networks. It finds that direct transfer fails, that action delay causes the largest drop, and that domain randomization often backfires. However, it models the transition gap as coarse parameter shifts and studies each gap in isolation. We keep its networks and gap taxonomy and add the time dimension of the perturbation plus combinations of gaps.
Understanding the performance of many TCP flows and PFDCT show the same failure in packet networks. Steady-state fluid reasoning misses global synchronization, the effect where many flows halve their sending rates at the same moment, and it misses incast bursts, where many senders converge on one receiver at once. Fixed marking thresholds collapse under these fast load spikes. Both papers are evidence that transient onsets, not average load, break controllers designed around equilibrium.
One strand constrains the design. Pantheon calibrates a five-parameter emulator to real paths with 17% average throughput and delay error. The parameters are static, so matching steady-state performance says nothing about fidelity to fast transients, and we do not treat calibration alone as sufficient. Mareček, Shorten and Yu show that controllers assuming static equilibrium responses from users trigger synchronized herding. This warns that estimators feeding back into routing can themselves create transients. We therefore measure network-level spillback incidence, not just per-intersection delay.
The method comes from the Osorio et al. double-queue model. It represents each road link as two coupled finite-capacity queues: one for vehicles waiting to leave, and one for free space propagating backward. It yields queue-length probability distributions and spillback probabilities through smooth difference equations. It was validated only on a single 100 m synthetic link, without control, noisy sensors or intersection coupling. We use it as a filter inside a controller on multi-intersection networks, which is new and carries real risk given the numerical instability the authors report under extreme congestion.
Research Program
The transient benchmark. For each Sim2Signal transition perturbation (reduced saturation flow from weather, heavier vehicles) and for demand surges, we define a time profile $\delta(t)$ that ramps to its peak over onset time $\tau$ and later recovers. We scale it so that its time average over the episode equals the constant shift $\bar\delta$ used in the static test:
$$\frac{1}{T}\int_0^T \delta(t)\,dt = \bar\delta,$$
where $T$ is the episode length. This matching ensures that any difference between static and transient results comes from temporal structure alone. We sweep $\tau$ over roughly two orders of magnitude around the median $T_r$, which we estimate per network by applying step changes in the calibrated simulator. We then add combined conditions that pair a transient transition shift with Sim2Signal's observation gap (detector dropout timed to coincide with onset) and action gap (actuation latency and countdown constraints).
The "real" side here is a held-out, independently perturbed high-fidelity simulation. This is Sim2Signal's own proxy protocol, not a street deployment, and we will state this plainly.
The spillback-belief controller. On each link, a double-queue filter takes noisy, delayed detector counts. It updates its arrival and service rates online by gradient descent on the likelihood of recent counts, which works because the model is differentiable. It outputs the current queue-length distribution and spillback probability, and these are appended to the signal policy's observation. For intersection coupling we use a simple node model that splits flow by turning ratio and limits it by the downstream free-space queue. Tractability comes from the queue model's closed-form transitions, not from differentiating through a microsimulator, which is exactly the bottleneck Oktay et al. describe for dynamic physics.
Baselines
The naive reference point is direct transfer. A DQN or PressLight controller is trained in the unperturbed calibrated network and evaluated unchanged on the transient benchmark, alongside the fixed-time and actuated NEMA dual-ring plans that ship with the Tempe and Bullhead City networks. Every method must at least beat the actuated plan under fast onsets, or learning adds nothing.
The strongest existing methods are the explicit-delay estimators that Sim2Signal found most effective, Delayed-Q and PRLight, together with its autoencoder-based state estimator. They are the right competitors because they embody the "estimation beats invariance" result our proposal builds on. The benchmark file reports no per-method numbers for them: I found no reported result on Lacuna for Delayed-Q, PRLight or per-gap degradation percentages on Sim2Signal. The only quantitative anchor it gives is that DQN and PressLight agreed on the best mitigation method 76% of the time. We will therefore rerun all baselines ourselves.
All methods share the 10 networks, identical training step budgets, the same hyperparameter search budget (a fixed number of trials per method, tuned on static-shift validation episodes only), and the same seeds. Our controller counts as beating the estimators if it reduces average travel time and spillback incidence in the regime $\tau < T_r$ on a majority of networks, with seed-level confidence intervals that exclude zero, while not losing in the quasi-static regime.
Two ablations isolate the key ingredient. The first replaces the double-queue filter with a steady-state belief of identical interface: queue estimated from a fluid model with a Bureau of Public Roads travel-time curve, recalibrated online the same way. This tests whether transient structure, rather than just having an extra estimated input, matters. The second freezes the filter's rates at their nominal values, which tests whether online recalibration matters. An oracle policy given the true perturbation parameters sets the ceiling.
Why not just…?
Why not just randomize the simulator's parameters during training? Sim2Signal found that domain randomization often performed worse than unmitigated transfer. It randomized static parameters, though, so it did not test randomization over time profiles. We therefore include temporal domain randomization, training over random onset times and surge shapes, as a baseline. This is a real threat. If it matches the belief controller at fast onsets, then the benchmark contribution stands but the estimator is unnecessary, and we will report it that way.
Why not just use a recurrent policy that learns to estimate the hidden state itself? A recurrent network sees the same detector history and could in principle infer spillback risk. We train a recurrent PressLight variant with a matched parameter count and budget. We expect it to underperform when detectors drop out during onsets, because it must learn queue physics from reward alone, whereas the double-queue filter encodes finite storage and backward-propagating space explicitly. The comparison in the combined-gap conditions settles this directly.
Why not just calibrate the simulator better, Pantheon-style? Calibration fits static parameters to observed average performance. The Pantheon emulator still carries 17% average error, and nothing in its five parameters describes onset speed. In our sweep, a well-calibrated static simulator corresponds exactly to the quasi-static training regime whose failure at small $\tau/T_r$ we are testing. If policies trained in a calibrated simulator show flat loss across $\tau$, the core claim is wrong.
Why not just train on real traffic logs, as the videoconferencing bandwidth challenge did? Offline learning on 18,859 real calls produced the winning model in the MMSys 2024 challenge. Signal control, however, lacks logs from several diverse controllers, and offline learning cannot evaluate actions outside the logged support. Even that challenge kept 9,405 emulated calls to obtain ground-truth capacity. Our transient benchmark is complementary: it supplies the controlled onsets that logs cannot label.
Proposed Experiments and Decisive Checks
The first check tests the timescale law. For each policy and network, we plot the performance loss relative to the unperturbed simulator against $\tau/T_r$. The claim predicts a knee near 1, and that constant-shift results match the large-$\tau$ end. If loss is flat in $\tau$, the stationarity gap does not exist and the project reduces to Sim2Signal as it stands.
The second check asks whether static tests rank mitigations the same way transient tests do. We compute the rank correlation between each method's standing under constant shifts and under fast onsets, using the 76% DQN–PressLight agreement as a reference for how stable rankings normally are. A low correlation would show that current robustness benchmarks mislead practitioners about which fix to deploy.
The third check targets compounding. It compares loss under joint onset, dropout and delay against the sum of the individual losses. A positive interaction term would give the first measurement of the compounding effect that Sim2Signal leaves open.
The fourth check is the controller comparison above, reported separately for $\tau < T_r$ and $\tau > T_r$. Metrics are travel time, throughput, spillback incidence (queue exceeding link storage), and Sim2Signal's emissions and conflict measures. We also report filter divergence rates under extreme density, because the Osorio model's known instability is the most likely way the method fails.
References
- Experimental and Numerical Investigation of 3D Printing PLA Origami Tubes under Quasi-Static Uniaxial Compression
- Neuromechanical Autoencoders: Learning to Couple Elastic and Neural Network Nonlinearity
- Sim2Signal: Sim-to-Real Benchmarks for Traffic Signal Control
- A differentiable dynamic network loading model that yields queue length distributions and accounts for spillback
- Understanding the performance of many TCP flows
- PFDCT: An Enhanced Transport Layer Protocol for Precise Flow Control in Data Centers
- Pantheon: the training ground for Internet congestion-control research
- Distributionally Robust Optimisation in Congestion Control
- ACM MMSys 2024 Bandwidth Estimation in Real Time Communications Challenge