Abstract
Spatiotemporal forecasting is hindered by complex heterogeneity and the "spatiotemporal mirage"—a phenomenon where short-term observations lead to inaccurate predictions due to insufficient historical context. We propose Spatial-Temporal-Decoupled Masked Autoencoders (STD-MAE), a self-supervised framework designed to capture robust representations by separately masking and reconstructing spatial and temporal dimensions. By leveraging long-range historical data, STD-MAE provides enriched context to downstream predictors. Experiments on PEMS traffic benchmarks demonstrate that STD-MAE achieves state-of-the-art performance and serves as a universal performance booster for existing forecasting architectures.
Introduction
Accurate spatiotemporal forecasting is vital for modern urban infrastructure. However, the data exhibits significant temporal heterogeneity (varying patterns over time) and spatial heterogeneity (varying patterns across locations). A critical challenge is the spatiotemporal mirage, where models with limited input windows misinterpret trends because they cannot distinguish between similar short-term patterns that lead to different long-term outcomes.
Methodology
The proposed STD-MAE framework decouples spatial and temporal learning to force the model to capture more meaningful representations.
Decoupled Masking Strategy
Unlike traditional joint masking, STD-MAE employs two distinct strategies:
- S-Mask: Randomly masks entire sensors to learn spatial dependencies and correlations between nodes.
- T-Mask: Randomly masks time steps to capture temporal periodicities and long-term trends.
Architecture
The framework utilizes dual Transformer-based autoencoders: Spatial-MAE (S-MAE) and Temporal-MAE (T-MAE). These encoders process long-range input sequences using patch embeddings and 2D positional encodings. The resulting representations are then integrated into downstream predictors, effectively augmenting them with long-term memory.
Results and Discussion
Evaluations on the PEMS03, PEMS04, PEMS07, and PEMS08 datasets show that STD-MAE consistently outperforms existing state-of-the-art models.
- Universality: Integrating STD-MAE features improved the performance of five base models (DCRNN, MTGNN, STID, STAEformer, and GWNet) across all metrics.
- Efficiency: The decoupled approach is computationally efficient, achieving a 72.5% speedup per sample on the PEMS07 dataset compared to non-decoupled methods.
- Optimal Masking: A 25% masking ratio was found to be most effective, balancing the difficulty of the reconstruction task with the inherent continuity of time-series data.
Conclusion
STD-MAE addresses the spatiotemporal mirage by decoupling spatial and temporal representation learning. This self-supervised approach provides a scalable and efficient way to incorporate long-range historical context into spatiotemporal forecasting models, leading to significant accuracy gains.
Summary updated