One Masked Model, Many Decoding Schedules: Compressing and Sharing Multi-Agent Plans

When several robots or drones coordinate, for example arriving at a target at the same instant, carrying an object together, or keeping a formation without collisions, each agent needs some picture of where the others intend to go. That picture has to travel over wireless links that are slow, lossy, and change with distance. This proposal asks one question. Can a single learned model of joint multi-agent motion serve three roles at once? It would act as the probability model that compresses those shared plans into short bitstreams. It would decode them in a small number of parallel steps instead of one symbol at a time. And it would fill in the parts of a plan that never arrived. Our bet is that a masked non-autoregressive generator can do all three. Such a generator produces a whole sequence in a few parallel rounds by repeatedly filling in masked positions, rather than predicting one step after another. Because it is trained to predict any subset of trajectory tokens from any other subset, it assigns a probability to every possible "decode these first, then those" order. Every such order is therefore a valid lossless entropy code, and the sender can pick one at transmission time. In the usual design the order is fixed once, at training.

The idea comes from noticing that two communities are solving the same problem under different names. Learned image codecs speed up decoding by splitting latent symbols into groups that are decoded together, as in the checkerboard's two passes. Masked generators speed up generation by unmasking groups of tokens together. In both cases the cost of parallelism comes from ignoring dependencies among symbols decoded in the same group. For a codec that cost shows up as extra bits. For a generator it shows up as incoherent output. Multi-agent temporal synchronization is the setting where both costs matter at once, and where packet loss makes a fixed decoding order fragile.

Core Claim

Consider quantized trajectory tokens $x_i$, indexed by agent and timestep. A decoding schedule $S$ partitions them into groups $G_1,\dots,G_K$ that are decoded in $K$ parallel passes. Under a model $p$, the expected code length of schedule $S$ is

$$ L(S) = \sum_{k=1}^{K} \sum_{i \in G_k} -\log_2 p\left(x_i \mid x_{G_1 \cup \dots \cup G_{k-1}}\right), $$

and the number of passes $K$ sets decoding latency. A single-pass schedule codes every token independently given shared history. A fully sequential schedule codes each token given all earlier ones. We make three claims. First, one masked model trained with random mask ratios gives competitive code lengths across the whole range of schedules. Codecs such as the Checkerboard Context Model and channel-sliced Transformer-based Transform Coding instead hard-wire one partition, a rigidity the literature already flags as a limitation. Second, choosing the schedule per transmission trades bits, passes, and robustness to loss against one another. Arithmetic coding needs the receiver to reproduce the sender's probabilities exactly, so a packet can only be decoded if its declared context arrived. Shallow schedules survive loss, and deep schedules save bits. The masked model also inpaints tokens that are undecodable or lost. Third, as a byproduct, the gap $L(S_1) - L(S_{\text{seq}})$ between one-pass and sequential code length is an exact measurement, in bits, of how much inter-agent dependency a single-pass decoder throws away. That measurement speaks directly to a live disagreement in the literature about whether one-shot decoding is enough for tightly coupled agents.

Where This Sits

Three strands of prior work each supply part of the method, and none combines them.

Parallel entropy decoding supplies the coding view. He et al.'s Checkerboard Context Model decodes half the latent grid from side information and the other half conditioned on the first, cutting Kodak decoding from 1323 ms to 29 ms with nearly unchanged rate. Hybrid Spatial-Temporal Entropy Modelling carries this to video, decoding within a frame in parallel while chaining frames strictly $t-1 \to t$. Both assume that every symbol the decoder conditions on has arrived and that one partition fits all inputs. We keep their core mechanism, coding groups conditioned on earlier groups, and drop both assumptions.

Masked non-autoregressive generation supplies the model. UniMTD trains a masked trajectory diffusion model that handles prediction, imputation, and recovery for many agents, and reports 0.18 s latency for 11 agents with 5 denoising steps. AutoBots decodes joint multi-agent futures in a single pass from learned seed queries at over 30 FPS. Neither produces a bitstream, and neither measures how much its parallel factorization costs in likelihood. Two papers constrain the design. Promises and Pitfalls of Generative Masked Language Modeling explains why decoding dependent tokens together produces incompatible outputs, which our code-length gap quantifies. Bringing Stability to Diffusion shows that masked-model training has higher gradient variance than autoregressive training, which is a real risk for an any-order model whose likelihoods must be sharp enough to compress well.

Temporal synchronization supplies the application and the failure model. Control Strategies for Multi-UAV Simultaneous Arrival under Communication Topology Changing reaches arrival-time consensus by exchanging scalar arrival estimates over links that drop and rejoin, and adds virtual sub-goals to keep agents in radio range. Guidance Laws for Multi-Agent Cooperative Interception coordinates approach angles as well as timing. Our motivation comes from these works: coordination beyond a single scalar needs richer shared state, and that state travels over exactly the links these papers describe. We do not replace their consensus laws. We supply the plan-sharing channel those laws would sit on top of.

Research Program

We would build three components. The first is a tokenizer. It quantizes each agent's motion into residuals relative to a constant-velocity extrapolation, using a fixed step size, so distortion is set by the step and all remaining coding is lossless over the resulting symbols.

The second is an any-order masked transformer over the agent-by-time token grid, with separate attention along time and across agents in the style of AutoBots. It is trained with uniformly random mask ratios and random mask shapes (per-agent, per-timestep, and scattered) so that its conditionals are calibrated for arbitrary contexts. We would track gradient variance during training. If it destabilizes training, we would adopt the variance-reduction ideas of Jia et al. before changing anything else.

The third is a schedule selector at the sender. Given the link's estimated loss rate and a pass budget $K$, it picks a partition. Candidates range from checkerboard-like splits over agents and time, through time-sliced splits, to greedy splits that put the most uncertain tokens in later passes. Each packet header declares which earlier groups its probabilities were conditioned on.

At the receiver, packets whose declared context arrived are decoded exactly with arithmetic coding. Tokens that could not be decoded are inpainted by running the same model as a generator. The analysis side has one task: measure $L(S)$ across schedules and scenes, and relate the one-pass gap to interaction density, such as the number of agents within collision range.

Baselines

The naive reference point is transmitting raw 32-bit float coordinates, together with the same trajectories quantized to our step and sent at a fixed number of bits per symbol. Every learned method must beat this. It is the de facto practice implied by AutoBots and UniMTD, which output dense floats.

The strongest simple competitor is classical predictive coding. It uses the same quantized constant-velocity residuals, coded with an adaptive arithmetic coder under a per-agent Laplace or histogram model with no cross-agent context. Trajectories are smooth, so this could capture most of the achievable compression. It gets the same quantization step and a tuned adaptation rate.

The strongest learned-codec competitors adapt He et al.'s checkerboard and Zhu et al.'s slice scheme to the agent-by-time grid. Each is trained as a dedicated model for its single fixed partition, with the same architecture size, data, and training steps as our model. These answer whether any-order training costs bits relative to a specialist. A fully sequential autoregressive model of the same size gives the best-rate reference, because it conditions every token on everything before it.

For generation quality, we compare against AutoBots and UniMTD on their own benchmarks. We do this to confirm that the any-order model is still a good forecaster, not to beat them there.

The key ablation holds the model fixed and swaps adaptive schedule selection for the single best fixed schedule. This isolates the value of choosing the order at transmission time. A second ablation removes inpainting and treats undecodable packets as lost. This isolates the value of the generator role.

"Beating" the baselines means three things. At equal pass count $K$, the proposal must reach lower code length than predictive coding. It must stay within a small margin of each specialist fixed-schedule codec. And under loss, it must dominate every fixed schedule on the curve of bits against recovered trajectory error. No reported result was found on Lacuna for learned compression of multi-agent trajectories, so every compression number here is new and must be produced by the baselines above.

Why not just…?

Why not just use delta coding with a standard entropy coder? This is the real threat, and we treat it as the main hurdle rather than a strawman. If per-agent residual coding lands within a few percent of our code length, then cross-agent context is not worth a transformer for rate alone. The proposal would then shrink to its diagnostic contribution, the code-length gap as a measure of interaction strength, plus the loss-recovery result. The comparison is run at identical quantization. We expect the gap to be largest in dense interactive scenes such as TrajNet++ synthetic and the sports data used by UniMTD, and smallest on sparse highway driving. We will report it split by scene density rather than as one average.

Why not just send scalar arrival estimates, as ETA consensus already does? For pure simultaneous arrival, a scalar is enough, and Tan et al. show consensus working that way. We concede this case. The proposal targets coordination that also depends on paths: collision-free convergence, multi-angle interception, and formation keeping. In those tasks an agent must know where others will be, not only when. The synchronization experiment therefore includes a scalar-only arm. Our method has to show fewer conflicts at matched bandwidth, not merely better arrival spread.

Why not just use a fixed checkerboard codec with retransmission or forward error correction? Retransmission adds round trips, and those round trips are exactly the latency that parallel decoding was meant to remove. Error-correcting redundancy spends bits uniformly, whether or not a given token is predictable from what already arrived. Our adaptive schedule instead spends redundancy selectively, and inpainting recovers the rest. Still, fixed checkerboard plus error-correcting codes tuned to the measured loss rate is a fair arm. We include it at matched total bits, and it could win at low loss rates, which would be worth knowing.

Proposed Experiments

The first experiment is the schedule sweep. On TrajNet++ synthetic, Basketball-U and Soccer-U, and Argoverse, we measure bits per agent-timestep as $K$ goes from one pass to fully sequential, for the any-order model and for each specialist fixed-schedule model. The decisive check is whether the any-order curve tracks the lower envelope of the specialists. If it falls well above that envelope, one model for all orders is too costly, and the flexibility claim fails.

The second experiment is the dependency ruler. We compute $L(S_1) - L(S_{\text{seq}})$ per scene and regress it on interaction density. AutoBots reports that adding social attention in the decoder cut predicted collisions by nearly 90% on TrajNet++ synthetic. If single-pass decoding with social attention were truly sufficient, the gap should be small on exactly that data. A large gap there would indicate that its one-shot decoder still leaves recoverable joint structure unexploited. We would also test whether scenes with large gaps are the ones where AutoBots' predicted collisions concentrate.

The third experiment is synchronization under loss. We rebuild the six-UAV, 400 km scenario from Tan et al., in which links drop at 40 s and rejoin at 70 s and consensus without sub-goals takes about 7,694 s, and extend it with a collision-avoidance and approach-angle requirement. Agents share compressed plans at a fixed bandwidth cap and at packet loss rates from zero to heavy. We report arrival-time spread, inter-agent conflicts, and bits sent. The comparison covers the scalar-only arm, fixed checkerboard with error-correcting codes, predictive coding, and our adaptive-schedule method with and without inpainting.

The proposal succeeds if adaptive scheduling dominates fixed schedules across loss rates and cuts conflicts relative to scalar-only exchange at matched bandwidth. It fails cleanly if predictive coding matches its rate and fixed checkerboard with error-correcting codes matches its robustness.