STAMImputer: Spatio-Temporal Attention MoE for Traffic Data Imputation

Yiming Wang, Hao Peng, Senzhang Wang, Haohua Du, Chunyang Liu, Jia Wu, Guanlin Wu

Introduction

Traffic data is the lifeblood of smart cities, enabling efficient routing, congestion management, and urban planning. However, real-world sensor networks are prone to failures caused by network outages, extreme weather, or hardware malfunctions. These issues often result in "block-wise" gaps—consecutive time steps or clusters of nearby sensors with missing data—that traditional imputation models struggle to reconstruct accurately.

Existing imputation methods typically suffer from two major limitations. First, they often rely on rigid pipelines that process temporal and spatial information in a fixed sequence. If a large block of time is missing, errors in the temporal module propagate through the entire network. Second, they frequently use static graphs based on fixed road maps. Since traffic relationships are highly dynamic (e.g., changing between peak and off-peak hours), static graphs fail to capture shifting global dependencies.

The STAMImputer Framework

To address these challenges, the authors introduce STAMImputer (Spatio-Temporal Attention Mixture of Experts Imputer). This framework dynamically balances temporal and spatial information using an adaptive routing mechanism.

The STAMImputer MoE framework architecture, showing the flow from input embeddings to imputation outputs.

Mixture of Experts (MoE)

The core of STAMImputer is an MoE framework where an Observation Expert acts as a smart router. By analyzing the sparsity and patterns of the input data, it determines the optimal weights for Temporal Experts (utilizing Multi-head Self-Attention, MSAT) and Spatial Experts (LrSGAT). This allows the model to prioritize spatial correlations when temporal data is missing in blocks, and vice versa.

Low-rank guided Sampling Graph ATtention (LrSGAT)

A key innovation within the spatial experts is the LrSGAT module. To capture global correlations without the prohibitive computational cost of a full global graph, STAMImputer employs a Sampling Projector.

The Sampling Projector mechanism within LrSGAT, used to identify significant nodes for efficient low-rank reconstruction.

The Sampling Projector identifies the most significant "traffic hub" nodes through Top-k sampling and combines them with probabilistic sampling to explore the network's "long tail." It then applies low-rank matrix factorization to reconstruct spatial relationships efficiently, turning the inherent redundancy of traffic data into a powerful tool for reconstruction.

Spatio-Temporal Representation

Before processing, data is transformed using a Discrete Wavelet Transform (DWT). This decomposes the signal into low-frequency "trends" and high-frequency "details," allowing the experts to focus on different layers of the traffic pattern for more nuanced imputation.

Experimental Results

STAMImputer was evaluated on four real-world traffic benchmarks: PemsD8, SZ-Taxi, DiDi-SZ, and NYC-Taxi. The researchers tested both Point Missing (random gaps) and Block Missing (consecutive gaps) scenarios.

Performance Highlights

  • State-of-the-Art Accuracy: STAMImputer consistently outperformed leading models such as ImputeFormer, SPIN, and SAITS. For instance, in the NYC-Taxi block missing scenario, it achieved a Mean Absolute Error (MAE) of 5.81, compared to ImputeFormer’s 6.14.
  • Robustness to Sparsity: The model's performance degraded significantly slower than competitors as missing data rates increased (up to 95%), validating the effectiveness of the MoE's adaptive weighting.
  • Downstream Impact: Imputed data from STAMImputer led to marked improvements in traffic prediction accuracy when used as input for models like Graph-Wavenet.

Key Insights and Future Directions

The success of STAMImputer highlights that dynamic balancing is essential for robust imputation. By treating different missing data patterns uniquely, the model avoids the pitfalls of fixed-sequence processing. Furthermore, the use of low-rank priors effectively leverages the natural redundancy in traffic networks.

While powerful, the construction of dynamic graphs remains computationally intensive. Future work will explore more efficient methods for generating these dependencies and investigate applying this MoE logic directly to real-time traffic forecasting.

Summary updated

Abstract

Traffic data imputation is fundamentally important to support various applications in intelligent transportation systems such as traffic flow prediction. However, existing time-to-space sequential methods often fail to effectively extract features in block-wise missing data scenarios. Meanwhile, the static graph structure for spatial feature propagation significantly constrains the models flexibility in handling the distribution shift issue for the nonstationary traffic data. To address these issues, this paper proposes a SpatioTemporal Attention Mixture of experts network named STAMImputer for traffic data imputation. Specifically, we introduce a Mixture of Experts (MoE) framework to capture latent spatio-temporal features and their influence weights, effectively imputing block missing. A novel Low-rank guided Sampling Graph ATtention (LrSGAT) mechanism is designed to dynamically balance the local and global correlations across road networks. The sampled attention vectors are utilized to generate dynamic graphs that capture real-time spatial correlations. Extensive experiments are conducted on four traffic datasets for evaluation. The result shows STAMImputer achieves significantly performance improvement compared with existing SOTA approaches. Our codes are available at https://github.com/RingBDStack/STAMImupter.
BibTeX
@inproceedings{wang2025stamimputer,
  author = {Yiming Wang and Hao Peng and Senzhang Wang and Haohua Du and Chunyang Liu and Jia Wu and Guanlin Wu},
  title = {{STAMImputer: Spatio-Temporal Attention MoE for Traffic Data Imputation}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/382},
}
Figure