Robots and software agents that share a task often talk over wireless links. When one of them moves out of range of its access point, its radio has to hand the connection over to a new one. During that switch the agent goes silent, typically for hundreds of milliseconds to a few seconds on ordinary hardware. That silence is rarely a problem for web browsing. It is a real problem for an agent that must work with teammates it has never met before and whose behavior it is trying to infer from what they do. Methods for inferring an unfamiliar teammate's type, the setting called ad hoc teamwork, assume that every observable event reaches the agent immediately. A handover blackout breaks that assumption. The simple fixes either discard the evidence the gap hides or treat the gap as noise, and both mean the agent needs many more observed partner events, and therefore more time, before it coordinates well.
This note proposes treating each blackout as a delayed, aggregated observation rather than as missing data. When the link comes back, the shared world has changed: a teammate has moved, items are gone, tasks are done. That change is evidence about what kind of teammate the agent is paired with. We score each hypothesis about the teammate by simulating it forward through the gap and asking how well it explains the state after the gap. During the gap, the agent plans against those same simulated teammates. It stops waiting for news. The claim can be tested directly. With realistic handover blackouts injected, a gap-aware estimator should identify teammate types in far fewer observed episodes than an estimator that ignores the gaps. It should also recover most of the team return that zero-latency inference achieves, and it should do this without infrastructure changes to the network.
Where the three directions meet
The wireless literature explains why blackouts exist and how long they last. Traditional handover disconnects first and then scans channels one by one. Scanning accounts for over 90% of the delay, as measured in Liu. Interactive voice needs handovers under 50 ms (Yao et al.), yet commercial stacks often take hundreds of milliseconds or more. Engineering can shrink the gap: sensor-guided pre-scanning reaches 23.45 ms (Liu), and station virtualization with software-defined rerouting reaches about 34.7 ms (Dely et al.). Both depend on coordinated infrastructure or custom drivers, which decentralized teams of strangers usually lack. We use these papers to calibrate the blackout distributions we inject, from near-ideal to legacy. We do not propose to beat them at networking.
Ad hoc teamwork supplies the core inference problem. OEATE (Yourdshahi et al.) keeps a population of hypotheses about a teammate's type and its continuous parameters, such as vision range. It updates the population only when the teammate completes a discrete task, and it plans with a tree search. Updating only on task completions is already a sample-efficiency choice. The file notes, however, that OEATE assumes an instant global signal whenever a task is completed. That is exactly the assumption a handover breaks, and it is the gap this proposal fills.
The latency-aware reinforcement learning (RL) line constrains the design. Delay-Aware Multi-Agent Reinforcement Learning shows that delays break the Markov property, the assumption that the current observation is enough to act on. It restores that property by adding the queue of in-flight actions to the state. It assumes fixed, known integer delays and teammates that were trained jointly, so neither stochastic blackouts nor unknown partners is covered. The sequence-model route also gets little support from the file. Benchmarking the Limits of In-Context Reinforcement Learning for Ad-Hoc Teamwork finds that transformers pretrained on 150 million transitions show flat in-context learning curves with new partners and sometimes block their teammates. That result motivates an explicit hypothesis-based estimator instead of hoping a recurrent policy infers the teammate implicitly. The handover-RL work of Handover Control in Wireless Systems via Asynchronous Multi-User Deep Reinforcement Learning optimizes the handover decision itself and treats users as parallel data collectors, not as teammates. It is complementary to this proposal rather than competing with it.
Core Claim
A handover blackout does not have to cost the agent its evidence about a teammate. Let $w(\theta)$ be the current weight of teammate hypothesis $\theta$, which specifies a type and its parameters. Let $s$ be the last world state observed before a blackout of $d$ steps, and let $s'$ be the first state observed after it. After reconnection, the weights are updated as
$$ w(\theta) \leftarrow w(\theta)\,\hat{P}(s' \mid s, \theta, d), \qquad \hat{P}(s' \mid s, \theta, d) = \frac{1}{K}\sum_{k=1}^{K} \kappa\big(\mathrm{sim}_k(s, \theta, d),\, s'\big), $$
Here $\mathrm{sim}_k$ is the $k$-th of $K$ stochastic forward simulations of a teammate of type $\theta$ for $d$ steps. The function $\kappa$ is a similarity kernel over the parts of the state the teammate can affect: its position, which items remain, and which tasks are done. The blackout length $d$ is itself only known after the fact, from timestamps.
Two consequences follow. First, task completions that happened inside the gap are credited to the right hypotheses rather than lost. Second, the same simulations serve as bridging predictions while the link is down, so the planner can avoid duplicating a task the teammate has probably already claimed.
A further point makes the problem specific to wireless teams rather than delay in general. Handovers happen at cell boundaries. A teammate's blackouts are therefore partly predictable from where it is. That lets the agent anticipate its partner's silences, which a fixed-delay model cannot represent.
Research Program
We will build three components on top of OEATE's hypothesis population and tree-search planner, and evaluate them under a shared blackout simulator.
The first component is the simulator. It overlays a grid of virtual access points on each environment and triggers a handover when an agent crosses a cell boundary. Each handover draws a blackout duration from one of three regimes. The ideal regime is tens of milliseconds, following the 23.45 ms and 34.7 ms results above. The fast-roaming regime is around 100–300 ms. The legacy regime reaches seconds, dominated by scanning. We convert these to environment steps at a fixed control rate and add jitter, because the delay-aware MARL paper's fixed-queue assumption is exactly what real handovers violate. During a blackout the affected agent receives no messages and no global task signals. Its own local sensing continues.
The second component is the gap-aware estimator. It has three pieces: timestamped, delayed delivery of task signals; the retroactive reweighting above, applied to OEATE's hypothesis bag, with crossover and mutation seeded by the survivors; and a small predictor, fitted from simulation, of when each teammate will next go silent given its position.
The third component is a planner that, during a blackout, branches over teammate rollouts drawn from the current hypotheses rather than freezing its model of the teammate at its last known state.
The analysis centers on sample efficiency measured in observed partner events and in episodes. For each blackout regime, we record how many episodes and how many delivered task completions each estimator needs before its top hypothesis matches the true teammate type and parameters within tolerance. We also record how much of the zero-latency team return it recovers. A secondary study moves to Overcooked layouts (Ruhdorfer et al.). There we replace the tree search with a best-response policy conditioned on the hypothesis posterior. The purpose is to check whether the idea survives in a denser coordination task with learned partners.
Baselines
The naive reference point is unmodified OEATE (Yourdshahi et al.) running under blackouts. Task signals that arrive during a gap are dropped, and the teammate model is frozen until reconnection. This is the baseline that shows how much handovers actually hurt. No reported result found on Lacuna for OEATE's numerical estimation error or task-completion rate on Level-based Foraging, so we will reproduce it at zero latency first and treat that reproduction as the ceiling.
The strongest existing competitor is a delay-aware variant built in the spirit of Delay-Aware Multi-Agent Reinforcement Learning. It pads the state with a buffer sized to the maximum blackout in each regime and delivers delayed signals in order with their true timestamps. We give this variant the timestamps, which the naive baseline does not have, so the comparison isolates retroactive simulation rather than mere awareness of delay. No reported result found on Lacuna for delay-aware MARL on these ad hoc benchmarks.
A learned competitor is a recurrent policy trained with online randomization over blackout durations. It receives a blackout flag and learns to cope from experience. It gets the same number of environment steps and the same hyperparameter-search budget, measured in GPU hours, as our method's simulation calls.
Three ablations isolate the key ingredients. The first removes retroactive reweighting but keeps bridging rollouts. The second keeps reweighting but freezes planning during gaps. The third removes the silence predictor.
All methods see identical partner pools, layouts, blackout seeds and episode counts. The number of hypotheses and planner rollouts is matched per decision. Beating the delay-aware competitor means a statistically significant gain across seeds in two respects. The method must reach correct type identification in fewer episodes in the fast-roaming and legacy regimes. It must also achieve higher recovered return, with at least half of the naive baseline's loss relative to zero latency closed in the legacy regime. On Overcooked, the reference point is the Overcooked Generalisation Challenge result, where the best-generalizing agents score 13 against an oracle's 216 (The Overcooked Generalisation Challenge). That setting varies layouts as well as partners, so we will report our fixed-layout results alongside it rather than claim to beat it.
Why not just…?
Why not just make the handover fast enough that it doesn't matter? The papers cited above show 23–35 ms is achievable with infrastructure support (Liu; Dely et al.). But those gains depend on coordinated access points, neighbor reports or custom station drivers. The file flags that dependence as the main barrier in decentralized ad hoc deployments. This is still a real threat for well-provisioned sites, and the ideal regime in our simulator exists to measure it. If gap-aware inference gives no benefit at tens of milliseconds, we will say so and scope the contribution to legacy and fast-roaming conditions.
Why not just use the delay-aware state buffer? Buffering in-flight actions fixes the Markov property for known, fixed delays among co-trained agents. It says nothing about re-scoring hypotheses on evidence the agent never saw arrive. The buffer-padded competitor above, which has timestamps, tests exactly this. If it matches our method, then retroactive simulation adds nothing and the claim fails.
Why not just train a recurrent policy with random delays and let it learn to infer the teammate? This is the most serious alternative, because domain randomization often works. Against it, Jing et al. find that even large pretrained sequence models do not infer new teammates in context. Blackouts make that inference harder, not easier. The equal-budget recurrent baseline settles the question empirically. If it matches our identification speed at equal environment steps, the explicit estimator is not worth its simulation cost.
Proposed Experiments
The primary testbeds are Level-based Foraging and Capture the Prey, following OEATE's settings of up to 15 agents on grids up to 45×45. We run each under the three blackout regimes plus a zero-latency control. Teammate types and parameters are drawn from OEATE's heuristic families.
The metrics are episodes and delivered task events until correct type identification, parameter estimation error over time, task-completion rate, recovered fraction of zero-latency return, and the rate of duplicated tasks and collisions during gaps.
The decisive check is a crossover analysis. We plot each method's identification speed against mean blackout length. The prediction is that the naive and buffer baselines degrade roughly in proportion to the fraction of events lost in gaps, while the gap-aware estimator stays near the zero-latency curve until blackouts become long enough that forward simulation diverges. We will measure that divergence point directly by varying the rollout count $K$.
A second check tests whether the predictability of handovers matters. We compare cell-boundary blackouts against blackouts placed at random times with the same duration distribution. If the silence predictor helps only in the first case, the advantage comes from structure specific to wireless handovers rather than from generic delay handling.
The Overcooked study reports the same metrics with learned teammates and serves as the test of whether the idea extends beyond planners over hand-designed teammate types.
References
- A Pre-Scanning Fast Handoff Scheme for VoIP in WLANs
- Secured Fast Handoff in 802.11-Based Wireless Mesh Networks for Pervasive Internet Access
- A software-defined networking approach for handover management with real-time video in WLANs
- On-line estimators for ad-hoc task execution: learning types and parameters of teammates for effective teamwork
- Delay-Aware Multi-Agent Reinforcement Learning
- Benchmarking the Limits of In-Context Reinforcement Learning for Ad-Hoc Teamwork
- Handover Control in Wireless Systems via Asynchronous Multi-User Deep Reinforcement Learning
- The Overcooked Generalisation Challenge
- The Overcooked Generalisation Challenge: Evaluating Cooperation with Novel Partners in Unknown Environments Using Unsupervised Environment Design