Traffic and sensor forecasting models increasingly let every sensor attend to other sensors across time. A careful audit of the standard benchmarks, however, finds that most of the predictive signal is each sensor's own past, and that a well-tuned seasonal linear model beats most graph neural networks. We propose a forecasting model that follows this finding. Each sensor attends densely to a long, multi-resolution summary of its own history. Attention to neighboring sensors is switched on only for sensors whose recent behavior a temporal-only predictor fails to explain, such as sensors near an accident or a sudden jam. The main design choice is that this on/off decision is made per GPU tile, the fixed-size block of the attention matrix that a GPU computes as one unit. A skipped neighbor interaction therefore costs nothing, and a kept one runs at full dense speed.
Why this matters: spatio-temporal attention models today face a trade-off. They can pay for all-to-all spatial attention at every step, most of which the audit suggests is noise. Or they can prune neighbors with index-and-mask operations that look sparse on paper but run no faster on a GPU, because the GPU still computes every tile that contains a single active entry. Our proposal makes the sparsity pattern come from three sources: the road graph, a fixed temporal hierarchy, and a residual signal. It lays tokens out so that this pattern coincides with tile boundaries by construction. The goal is a model whose spatial compute scales with how many sensors are currently surprising, not with the square of the sensor count, and whose history can span days rather than the usual one hour.
Core Claim
A joint space-time attention can combine three pieces: a node-major, tile-sized multi-scale history per sensor; graph-restricted neighbor blocks that include time lags; and a gate that turns neighbor blocks on or off from temporal residuals. We claim this model matches or beats the strongest temporal-first hybrid on overall error, and clearly beats it during high-residual (non-recurring) periods. We further claim it executes on a stock block-sparse FlashAttention-style kernel with tile occupancy close to one. The alternative is to compose sparsity from graph pruning and flattened space-time ordering. That alternative, as we will show, leaves most executed tiles mostly empty.
To make occupancy precise, let $M$ be the binary attention mask, $b$ the tile width, and $\mathcal{T}(M)$ the set of tiles containing at least one allowed entry. Tile occupancy is
$$\rho(M) = \frac{\mathrm{nnz}(M)}{b^2\,|\mathcal{T}(M)|},$$
that is, the fraction of computed entries that the model actually uses. Wall-clock cost tracks $|\mathcal{T}(M)|$, not $\mathrm{nnz}(M)$. This is the "tiling tax" measured by Efficiently Dispatching Flash Attention For Partially Filled Attention Masks. Our design targets $\rho \approx 1$ and a small $|\mathcal{T}(M)|$ at the same time.
How the Pieces Fit
The three directions play different roles. Hardware-unfriendly sparsity is the constraint that shapes the design. Spatio-temporal graph forecasting is the application, and its benchmark audit is the main evidence behind the design. Hierarchical temporal modeling supplies the representation that makes long histories affordable.
On the hardware side, several papers supply the execution method and the cautionary results. Efficiently Dispatching Flash Attention For Partially Filled Attention Masks builds a coarse binary tile matrix, skips empty tiles, and reorders indices with Reverse Cuthill-McKee, a classic graph-bandwidth reduction. It does this only for static one-dimensional masks. We reuse its kernel strategy but design the mask so that reordering becomes a minor factor. When to Use Efficient Self Attention? finds that sparse attention variants lose to dense attention below roughly 1,750–2,000 tokens. This is a warning that sparsifying attention over only 207 sensors is pointless. We therefore apply sparsity only where the flattened sensor-by-history token count reaches hundreds of thousands. AdaSplash-2 beats FlashAttention-2 only above about 60% block sparsity, which tells us which sparsity regime our gate must reach to pay off. Video-BLADE and FG-Attn show that regular video grids can be reordered or gathered into tile-friendly form, using a space-filling curve and an asynchronous gather respectively. Neither handles an irregular sensor graph.
On the forecasting side, the audit plays the motivating role. A Critical Audit of Spatiotemporal Forecasting Benchmark Datasets and Baselines shows that on METR-LA and PEMS-BAY, spatial information adds little once each node's autocorrelation is modeled. It also shows that default first-differencing erases low-frequency signal, and that a GNN trained on SARIMA residuals sets the best reported error. TESTAM points the same way from inside deep learning. It routes each node among an "identity" expert with no spatial input, a static-graph expert, and an attention expert, and the identity expert often wins. Two differences separate our design from TESTAM. First, our routing unit is a GPU tile driven by a temporal residual, rather than a soft expert mixture computed in standard framework ops. Second, our temporal context is a multi-day hierarchy rather than a short flat window. DSSA-TCN selects top-$k$ spatial neighbors to reach $O(kN)$ cost, but implements the selection with ordinary indexing. It also keeps a flat 12-step history. We keep its idea that neighbors should be restricted, and change both how the restriction is executed and how far back the history reaches. DynST prunes input data rather than attention and reports up to 1.721x speedup for STGCN on WeatherBench. Its zeroed inputs still flow through dense backbones, and quality degrades past 60% sparsity.
On the hierarchy side, StreamForest shows that a fine recent window plus a merged long-term event memory keeps 96.8% of accuracy under 8x token compression. Its dynamically restructured tree, however, relies on irregular pointer manipulation that works against static GPU tiling. Multi-tier traffic hierarchies such as HiSTGNN and GEnSHIN aggregate time and space at several scales but run as dense framework operations. We borrow StreamForest's split between a fine recent window and coarse memory. We deliberately give up its adaptive merging in exchange for a fixed geometric hierarchy whose level sizes equal the tile width.
Research Program
The token layout is the core artifact. Each sensor gets $L$ temporal levels of exactly $b$ tokens each, with $b = 64$, the tile width. The first level holds the last $b$ five-minute readings. Each deeper level summarizes the previous history at a coarser stride through learned pooling over contiguous spans; for example, the strides could be 30 minutes, 3 hours, and one day at the same time of day. Tokens are ordered node-major: sensor, then level, then slot. Every sensor's own history is then an $Lb \times Lb$ block made of whole tiles. Intra-sensor attention is dense and fully aligned no matter how the sensors are numbered.
Neighbor attention connects sensor $i$ to sensors $j$ within $h$ hops on the road graph. On the fine level, it uses a lag band, so that a query at time $t$ sees neighbor $j$'s fine tokens up to a travel-time-sized offset; this lets congestion waves propagate. On the coarse levels, the connection is dense. Because each (sensor, level) pair occupies whole tiles, every neighbor interaction is a small set of whole tiles. The only partially filled tiles come from the lag band. We will measure $\rho$ directly rather than assume it.
The gate is where the audit's finding enters the design. A per-sensor temporal-only predictor is trained first: either SARIMA or the model's own intra-sensor branch with neighbor tiles disabled. At inference time, sensor $i$'s neighbor tiles are activated only if its recent residual exceeds a threshold $\tau$. Formally, row block $i$ of the tile matrix includes neighbor tiles iff
$$\frac{1}{w}\sum_{s=t-w+1}^{t} |r_i(s)| > \tau,$$
where $r_i(s)$ is the temporal-only one-step residual at time $s$ and $w$ is a short window. During training we replace the hard threshold with a straight-through estimator and add a penalty on the expected number of active tiles. This lets $\tau$ trade accuracy against compute. The resulting tile matrix is rebuilt per batch and handed to a binary-block-masked kernel in the style of Efficiently Dispatching Flash Attention For Partially Filled Attention Masks. Where GPU support allows, we will also compare against an α-entmax variant from AdaSplash-2, a softmax replacement that assigns exactly zero weight to some entries, to test whether learned within-tile sparsity adds anything once the gate has already removed whole tiles.
The analysis program asks three questions. Does direct lagged cross-sensor attention help beyond what the temporal hierarchy alone provides? Does any benefit concentrate in high-residual periods? And does executed spatial compute actually scale with the number of gated-on sensors?
Baselines
The naive reference points are two. One is a seasonal-naive forecast (the value one week earlier). The other is a tuned per-sensor SARIMA on undifferenced data, following the audit's protocol. Any learned model that cannot beat SARIMA has no case, and the audit shows this failure is common.
The strongest accuracy competitor is the audit's hybrid: SARIMA followed by a GCRN-GRU trained on the residuals, from A Critical Audit of Spatiotemporal Forecasting Benchmark Datasets and Baselines. It reports MAE 3.32 on METR-LA. It is the strongest competitor because it embodies the same "temporal first, spatial only for residuals" logic as our gate, but without long hierarchical history or tile-level execution. We will rerun it with the audit's preprocessing (no default differencing), the same train/validation/test splits as our model, and the same hyperparameter-search budget in GPU-hours. Beating it means lower MAE averaged over horizons 3, 6, and 12 on METR-LA and PEMS-BAY, confirmed across three seeds, together with a larger margin on the top residual decile.
For PEMS-BAY, the reported reference is DSSA-TCN at MAE 1.53 and RMSE 3.50. The file does not state which horizon these numbers correspond to, so we will rerun DSSA-TCN ourselves rather than compare against the headline figure. TESTAM is the closest competitor in mechanism and is run on the same splits. No per-horizon result was found on Lacuna for TESTAM, so it too is rerun. Its reported cost advantage (roughly 2x cheaper to train than GMAN) makes it the efficiency reference among published models.
The efficiency baseline that matters most is a factorized dense model. It uses the same multi-scale history, but runs per-sensor temporal attention and then all-sensor spatial attention per slot, both through dense FlashAttention. It has the same parameter count and the same token hierarchy. This is the architecture a skeptical engineer would build instead.
Three ablations isolate the proposal's ingredients. The gate is forced always-on, which gives graph-restricted joint attention, and forced always-off, which gives a purely temporal hierarchical model analogous to TESTAM's identity expert. The node-major layout is replaced by a time-major flattening with Reverse Cuthill-McKee sensor ordering, to test whether our layout, not reordering, is what delivers occupancy. The hierarchy is replaced by a flat 12-step window, to test whether long history is where the gains come from.
Why not just…?
Why not just use SARIMA plus a residual GNN, as the audit recommends? This is the most serious threat on the standard benchmarks. If spatial signal is truly marginal, our model's extra machinery may buy nothing on average. Our design takes the audit seriously rather than arguing with it. The gate is essentially a learned, tile-executable version of the audit's recipe. What we add is multi-day history per sensor and lagged neighbor paths during surprising periods. If the hybrid matches us overall and in the high-residual decile, we will report that the hierarchy and gating add nothing over a linear-plus-residual design. That would be a useful negative result for this literature.
Why not just use factorized dense attention with FlashAttention? This is the real hardware threat. For 207–883 sensors, dense spatial attention per slot sits below the tipping point reported by When to Use Efficient Self Attention?, and it is fast and simple. The factorized model cannot, in one layer, let sensor $i$ at time $t$ see sensor $j$ at time $t - \delta$. It also pays full spatial cost at every slot, even when almost every sensor is predictable. The experiments settle this in two ways. Accuracy is compared at matched parameter count, with high-residual periods scored separately. Wall-clock time is compared as the sensor count grows from METR-LA to EXPY-TKY's 1,843 links. If factorized dense wins on both at every scale, the joint gated design is not justified, and we will say so.
Why not just prune neighbors with top-k as DSSA-TCN does, or prune inputs as DynST does? Both reduce arithmetic on paper. Unless the survivors fill whole tiles, however, they leave executed tiles mostly empty or ragged tensors that need padding. That is exactly the gap between reported sparsity and actual speed documented by Efficiently Dispatching Flash Attention For Partially Filled Attention Masks and acknowledged by DynST. We measure this directly by reporting $\rho$ and executed tile counts for DSSA-TCN-style masks laid out in our kernel, next to wall-clock time.
Why not just learn an adaptive event tree as StreamForest does? Adaptive merging preserves information well, but its restructuring is hard to map onto static tiles. Traffic periodicity also makes fixed strides a reasonable prior. The flat-window and fixed-hierarchy ablations show how much long history contributes. If a fixed hierarchy recovers most of the gain, the extra cost of adaptive trees is not warranted here.
Proposed Experiments
Data comes from METR-LA and PEMS-BAY, where the audit's claims apply; PEMS07 with 883 sensors, as used by DSSA-TCN; and EXPY-TKY with 1,843 links, as used by TESTAM. Preprocessing follows the audit, with no default differencing. Histories span one to two weeks, which the hierarchy compresses to $L \cdot 64$ tokens per sensor. We report MAE, RMSE, and MAPE at 15, 30, and 60 minutes. We also report the same metrics restricted to test windows in the top decile of SARIMA residual magnitude, which serves as a proxy for non-recurring events because no incident labels exist in these benchmarks. Efficiency is reported on one GPU model and, if available, a second architecture to check kernel portability. Efficiency metrics are forward-pass latency, training throughput, peak memory, executed-tile fraction, and $\rho$.
Three checks decide the question. First, the gate curve: sweeping $\tau$ from always-off to always-on should trace a curve where most of the always-on accuracy on high-residual windows is recovered while only a minority of neighbor tiles are active. If the curve is flat, spatial attention is not doing work, and the audit is confirmed even for long-history models. Second, the scaling check: executed spatial cost should grow with the number of gated-on sensors, not with $N^2$. The crossover against factorized dense attention should appear somewhere between PEMS07 and EXPY-TKY. If no crossover appears even at 1,843 sensors, the hardware argument fails. Third, the layout check: node-major hierarchical layout should reach $\rho$ near one with or without Reverse Cuthill-McKee ordering, while time-major flattening should need reordering and still fall short. This would show that alignment comes from the layout design and not from a reordering heuristic applied afterward.
References
- Efficiently Dispatching Flash Attention For Partially Filled Attention Masks
- When to Use Efficient Self Attention? Profiling Text, Speech and Image Transformer Variants
- AdaSplash-2: Faster Differentiable Sparse Attention
- BLADE: Block-Sparse Attention Meets Step Distillation for Efficient Video Generation
- FG-ATTN: LEVERAGING FINE-GRAINED SPARSITY IN DIFFUSION TRANSFORMERS
- A Critical Audit of Spatiotemporal Forecasting Benchmark Datasets and Baselines
- TESTAM: A Time-Enhanced Spatio-Temporal Attention Model with Mixture of Experts
- DSSA-TCN: Exploiting adaptive sparse attention and diffusion graph convolutions in temporal convolutional networks for traffic flow forecasting
- DynST: Dynamic Sparse Training for Resource-Constrained Spatio-Temporal Forecasting
- StreamForest: Efficient Online Video Understanding with Persistent Event Memory
- HiSTGNN: Hierarchical Spatio-temporal Graph Neural Networks for Weather Forecasting
- GEnSHIN: Graphical Enhanced Spatio-temporal Hierarchical Inference Network for Traffic Flow Prediction