Agents in a cooperative team usually see only part of the world. To act well, each one has to remember what it saw earlier: an enemy that walked out of view, a teammate that went behind a wall. The cheapest way to give an agent a detailed memory is to feed every observed object, at every timestep, into one long sequence and run a linear-time sequence model over it. A state space model (SSM) such as Mamba is the standard choice. It is a recurrent network whose hidden state is updated token by token at cost linear in sequence length. That design has a hidden flaw that vision researchers already know about. Sequence models read tokens in one direction, so any token is blind to the tokens that come after it. In time this is correct, because an agent must not see the future. Within a single moment it is wrong: the enemy listed fifth cannot "see" the enemy listed sixth, even though both are in view right now.
We propose to build the sequence model so that it is causal across timesteps and fully two-way within each timestep. Within a timestep, the observed objects are ordered by their bearing around the agent rather than by arbitrary ID. This keeps the cost linear in (timesteps × objects), and it keeps a per-object memory instead of squeezing each frame into one vector. The causal-spatial mismatch literature is the mechanism we import. Decentralized multi-agent reinforcement learning (MARL) under partial observability is where we apply it. The requirement that execution be strictly causal in time is the constraint that shapes the design.
Core Claim
When a causal linear sequence model reads a flattened stream of entity tokens (one token per visible unit, per timestep), it inherits the "causality curse." Vision papers describe this curse for raster-scanned images: MambaIRv2 names it, and Transfusion shows that causal masks over image patches hurt quality. In MARL the curse is worse, because partial observability changes which entities are visible and therefore their positions in the scan from one step to the next. We predict a specific signature. Per-entity outputs, such as the value of attacking a particular enemy in StarCraft, should be least accurate for entities that come late in the within-step order. That error should grow as sight range shrinks.
The fix is a block-causal scan. Each timestep is one block. Inside the block, a forward and a backward scan over the bearing-ordered entities let every entity see every other entity at that moment. Only the forward state at the end of the block is carried to the next timestep. If $x_{t,k}$ is the $k$-th entity token at step $t$ and $h_{t-1}$ is the state carried from the previous step, the forward pass is
$$\overrightarrow{h}_{t,k} = \bar A_{t,k}\,\overrightarrow{h}_{t,k-1} + \bar B_{t,k}\,x_{t,k}, \qquad \overrightarrow{h}_{t,0} = h_{t-1},\quad h_t = \overrightarrow{h}_{t,N_t},$$
where $\bar A, \bar B$ are Mamba's input-dependent transition and input matrices and $N_t$ is the number of entities visible at step $t$. A backward pass $\overleftarrow{h}_{t,k}$ runs over the same block starting from $h_{t-1}$. The output for entity $k$ reads both $\overrightarrow{h}_{t,k}$ and $\overleftarrow{h}_{t,k}$. Nothing from step $t+1$ ever reaches step $t$, so the policy remains executable by a decentralized agent.
Relation to Prior Work
The vision literature supplies the ingredients. PlainMamba removes row jumps with a continuous zigzag scan and adds direction-aware parameters so the model knows which way it is sweeping. VmambaIR runs six directional scans. Non-causal selective state space model for image restoration shuffles tokens to break directional bias. Spectral VMamba orders patches by graph-Laplacian eigenvectors to get a content-driven, rotation-robust order. All of them assume a static image on a fixed grid, and none has a time axis whose causality must be preserved.
Our bearing order is a cheap, non-grid analogue of their orderings. Agents observe a variable set of entities rather than a lattice, which is the open problem the file lists as "generalization to non-grid topologies." We borrow PlainMamba's direction-aware parameters for the forward and backward passes. We do not use the spectral ordering, because its per-step eigendecomposition is too costly at simulator rates.
Three MARL lines come close but avoid the question.
Sports-Traj puts a Transformer over agents in space and a bidirectional Mamba over time. Its bidirectional time scan is fine for offline imputation but would leak the future in control, and it assumes a fixed number of agents. STAIRS-Former uses recursive spatial attention plus a hierarchical GRU in offline MARL. It is the work that motivates us most directly, because it reports that a single pooled history token is "lossy memory," which is exactly what an entity-level stream avoids. KalMamba supplies the method: parallel scans that keep SSM training cost flat in sequence length under partial observability. It is single-agent and purely temporal.
GlobeDiff and SIDE attack partial observability by inferring a global state, using diffusion and a variational graph autoencoder respectively. Our work is orthogonal: it changes the agent's own memory, not the training-time state estimate.
SMACv2 constrains the evaluation. It shows that original SMAC can be won by open-loop policies, so any memory claim must be tested under procedural generation.
Research Program
We will build the block-causal entity SSM as a drop-in history encoder for MAPPO and QMIX actors. Both live in a PyMARL-style codebase; pymarl_HDA is the candidate starting point. Each visible ally and enemy becomes one token holding its observed features plus a learned entity-type embedding. Attack-action values are read per enemy token, and movement values are read from a pooled summary.
Bearing is computed from the egocentric relative offset. The sweep starts at the agent's last movement direction, so the order is defined for any number of entities and does not depend on unit IDs. A unit leaving view simply drops out of the block. If it reappears, the model must match it to its earlier tokens through features and the carried state.
The analysis has two parts. The first is a diagnostic. In the naive flattened causal model, we measure attack-value error and target-choice accuracy as a function of an enemy's position in the within-step order. Ground truth comes from a centrally trained critic with full state, evaluated on held-out trajectories. We repeat this across sight ranges. The second is a memory probe. We train a linear readout from the carried state to the last-seen position of an enemy that left view $k$ steps ago, and compare entity-stream memories against pooled-GRU memories.
Baselines
The naive reference point is the standard MARL recipe: a per-step MLP or attention encoder that pools entities into one vector, followed by a GRU over time. This is what the SMACv2 and COMRAD baselines use. It defines "do nothing."
The strongest existing competitor on this exact question is factorized attention plus recurrence, modeled on STAIRS-Former: full self-attention among the entities at each step, followed by a GRU. It is non-causal in space and already handles variable entity counts, so it directly tests whether our within-step scan adds anything over attention.
Above it sits a causal space-time Transformer, in which every entity token attends to all entity tokens at the current and earlier steps. It gives entity-level memory at quadratic cost in (timesteps × entities) and serves as the quality ceiling we hope to approach at linear cost.
Two ablations isolate the key ingredient. The flattened causal Mamba with ID order is the model that should exhibit the mismatch. A block-causal model with random within-step order, in the spirit of the token-shuffling approach, separates the contribution of two-way scanning from that of bearing order. Where relevant, we also report GlobeDiff-augmented MAPPO as a reference for state-inference approaches.
All models will use the same learner and environment steps. Encoder parameter counts will match within 10%, and each model gets the same hyperparameter search of about 20 configurations on a held-out map. We run at least five seeds and report interquartile means.
Reported numbers are sparse. The SMACv2 paper states that QMIX and MAPPO win rates "often fall below 25%" on 20-vs-23 maps. On SMAC-v1 6h_vs_8z with sight range reduced to 3, GlobeDiff reports a win rate of about 0.47 against ≤0.12 for vanilla MAPPO. No per-map SMACv2 number for STAIRS-Former or GlobeDiff appears in the file, so we will rerun both. Beating the strongest baseline means a seed-robust win-rate gain over factorized attention plus GRU on the 20-vs-23 maps under extended partial observability, or matching the space-time Transformer at a clearly lower wall-clock cost.
Why not just…?
Why not just use attention over entities at each step and a GRU over time? This is the real threat. With a few dozen entities, per-step attention is cheap, so the efficiency argument for a spatial scan alone is weak. Our claim does not rest on spatial cost. It rests on memory granularity. A pooled GRU keeps one vector per step, and STAIRS-Former reports this is lossy under partial observability, whereas an entity stream keeps each entity's history addressable. If the factorized baseline matches us on win rate and on the re-appearing-enemy probe, the entity-level memory hypothesis is wrong. We would then report that negative result rather than reframe it. If a hybrid with attention within steps and the block-causal stream across steps does as well as our bearing scan, we adopt the hybrid and drop the scan.
Why not just run bidirectional or multi-directional scans, as vision SSMs do? Over the whole history, a backward scan reads the future and cannot be executed by a decentralized agent. Within a step, two-way scanning is exactly what we do. The new part is the block structure that keeps time causal, and the random-order ablation tests whether bearing order matters beyond two-way scanning. More scan directions, as in VmambaIR, multiply cost by the number of passes. Unlike images, a set of entities has no natural second axis to sweep along.
Why not just use a causal space-time Transformer? It is the cleanest solution and our quality ceiling. Its cost grows with the square of (timesteps × entities). At SMACv2's 20-vs-23 scale with long histories, this should dominate rollout and training time; KalMamba's flat-cost scans are the reason we expect the SSM to scale better. We will measure throughput directly rather than assume it.
Why not just infer the global state, as GlobeDiff or SIDE do? Those methods improve the centralized estimate, but the decentralized actor still needs its own memory at execution time. GlobeDiff also adds iterative sampling. The two approaches compose, and we test the combination rather than treat them as rivals.
Proposed Experiments
The main testbed is SMACv2 with procedurally generated start positions and teams, under extended partial observability, on the protoss, terran and zerg 10-vs-11 and 20-vs-23 maps. We add a secondary check on SMAC-v1 6h_vs_8z with sight range 3, so we can compare against GlobeDiff's reported setting. Metrics are win rate, per-enemy target accuracy, memory-probe error, and environment steps per second.
Four outcomes would be decisive. First, the flattened causal model should show target accuracy that falls with within-step position, and this slope should steepen at shorter sight ranges. If no slope appears, the mismatch does not bite in MARL and the premise fails. Second, the block-causal model should remove that slope. Third, it should beat factorized attention plus GRU on 20-vs-23 win rate, with the advantage concentrated in episodes where enemies leave and re-enter view. Fourth, its throughput should stay roughly flat as history length grows while the space-time Transformer's falls.
Extending to egocentric pixels in COMRAD is deliberately deferred. Its reported failures, including the unsolved Lava Maze signaling task, may stem from credit assignment rather than memory, and we do not want to confound the two questions.
References
- MambaIRv2: Attentive State Space Restoration
- Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model
- PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognition
- VmambaIR: Visual State Space Model for Image Restoration
- Non-causal selective state space model for image restoration
- Spectral State Space Model for Rotation-Invariant Visual Representation Learning
- Sports-Traj: A Unified Trajectory Generation Model for Multi-Agent Movement in Sports
- STAIRS-Former: Spatio-Temporal Attention with Interleaved Recursive Structure TransFormer for Offline Mulit-task Multi-agent Reinforcement Learning
- KalMamba: Towards Efficient Probabilistic State Space Models for RL under Uncertainty
- GlobeDiff: State Diffusion Process for Partial Observability in Multi-Agent System
- SIDE: State Inference for Partially Observable Cooperative Multi-Agent Reinforcement Learning
- SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learning
- COMRAD: A Benchmark for Embodied Cooperative Multi-Agent Reinforcement Learning