WaveDiST: A Wavelet Diffusion Transformer for Spatio-Temporal Estimation on Unobserved Locations

Huiling Qin, Yuanxun Li, Weijia Jia

TL;DR

This paper introduces WaveDiST, a novel approach for estimating spatio-temporal data in unobserved urban areas. By combining wavelet decomposition with a diffusion transformer, WaveDiST effectively handles sparse and noisy sensor data, outperforming existing methods in various urban sensing applications.

What Problem Does This Paper Address?

Urban areas are increasingly monitored using sensors for traffic, environmental quality, and economic activity. However, due to cost and logistical constraints, many locations remain unobserved, lacking historical data for direct estimation. This poses a significant challenge for urban planning, real-time operations, and anomaly detection. Traditional imputation methods struggle because they often rely on historical data, which is precisely what's missing for these unobserved areas. The problem is compounded by sensor data sparsity, noise, and the complex, heterogeneous spatio-temporal patterns inherent in urban dynamics.

Methods

WaveDiST tackles this by employing a multi-stage approach:

  1. Wavelet Decomposition: The core idea is to decompose the spatio-temporal data into high-frequency and low-frequency components using wavelet transforms. This separation is crucial because sensor noise tends to be concentrated in high-frequency signals, while gradual trends and overall patterns are captured by the low-frequency component.
  2. Diffusion in High-Frequency Space: A denoising diffusion probabilistic model is applied only to the high-frequency components. This allows the model to learn to reconstruct detailed variations while simultaneously denoising the noisy signals. The diffusion process is guided by the low-frequency component.
  3. Conditional Diffusion Transformer: A transformer architecture is used to process the denoised high-frequency components. This transformer is conditioned on:
  4. Low-frequency component: This provides global spatio-temporal profiles and slow trends, acting as a stable guide for reconstruction.
  5. Denoising step embeddings: These inform the model about the current stage of the diffusion process.
  6. Spatio-Temporal Attention Module: Within the transformer, specialized attention mechanisms are employed to capture interdependencies:
  7. Temporal Attention: Models pairwise temporal interactions between time steps.
  8. Spatial Attention: Incorporates graph attention to learn node transfer patterns based on spatial relationships, crucial when historical temporal data is absent.

This combination allows WaveDiST to reconstruct unobserved data by leveraging both the detailed variations from denoised high-frequency signals and the overarching trends from the low-frequency component, all within a robust transformer framework.

Datasets

The paper evaluates WaveDiST on five benchmark datasets from various urban domains:

  • Jinan Taxi: Traffic data.
  • Jinhua Sales: Economic data.
  • Beijing Air: Environmental data (PM2.5 concentrations).
  • METR-LA: Traffic data.
  • PEMS08: Traffic data.

These datasets are commonly used for evaluating spatio-temporal imputation performance and represent diverse urban sensing scenarios.

Key Insights and Ideas

  • Decomposition is Key: Separating data into high and low frequencies is a powerful way to disentangle noise from underlying trends, making imputation more robust.
  • Diffusion for Denoising and Reconstruction: Applying diffusion models to the high-frequency components effectively denoises them while the model learns to reconstruct the original signal, a more challenging task than just denoising.
  • Low-Frequency as a Stable Anchor: Using the low-frequency component as a condition for the diffusion process provides a stable global trend, preventing the model from drifting and improving reconstruction accuracy.
  • Spatio-Temporal Attention for Unobserved Data: The combination of temporal and spatial attention is critical for inferring information about unobserved locations, as it allows the model to learn how data evolves over time and how different locations relate to each other.
  • Transformer's Adaptability: The transformer architecture, with its attention mechanisms, is well-suited to handle the complex and varying spatio-temporal dependencies in urban data.

Key Results

  • Superior Performance: WaveDiST consistently outperforms state-of-the-art baselines across all five benchmark datasets, achieving significant improvements in MAE (Mean Absolute Error). For example, it shows a 9.8% MAE improvement over STEMGNN on Jinan Taxi and 20.9% on Jinhua Sales.
  • Robustness to Missing Data: The model demonstrates remarkable robustness to increasing missing rates. Even at a 70% missing rate, WaveDiST's accuracy degrades minimally (around 10%) and often remains superior to baselines operating at much lower missing rates.
  • Ablation Studies Confirm Contributions: Ablation studies highlight the importance of each component: removing diffusion significantly degrades performance, while removing wavelet transform or the low-frequency condition also leads to reduced accuracy, indicating the synergistic effect of the proposed modules.
  • Qualitative Improvements: Visualizations show WaveDiST effectively estimates traffic speeds and PM2.5 concentrations, capturing both gradual trends and sudden fluctuations while maintaining spatial continuity and smoothing out noise.

Limitations and Future Work

  • Wavelet Basis Sensitivity: While the paper notes that WaveDiST is not highly sensitive to the choice of wavelet basis, exploring a wider range of bases or adaptive wavelet selection could be a future direction.
  • Computational Cost: Diffusion models, while powerful, can be computationally intensive. Future work could focus on optimizing the diffusion process or exploring more efficient transformer architectures.
  • Domain-Specific Constraints: The paper suggests opportunities for more efficient architectures and better domain-specific constraint incorporation. While WaveDiST incorporates low-frequency trends, further integration of explicit domain knowledge could enhance performance in specific applications.

Summary updated

Abstract

Spatio-temporal estimation plays a vital role in numerous scientific and engineering tasks, particularly for novel or unobserved locations lacking historical references. Many areas remain unobserved by sensors due to their non-core location or pending development status. The states of these areas can only be estimated through similar nodes in the geospace, rather than through historical data with temporal trends. Estimating these unobserved node states is crucial for city-wide spatio-temporal sensing and urban development, extending beyond simple point or block data imputation. In this study, we introduce a diffusion point process in high-frequency space to develop a robust spatio-temporal diffusion transformer for urban estimation where partial historical reference data is lacking. Our approach decomposes spatio-temporal data into high and low-frequency components through wavelet transform, and trains a diffusion model of spatial temporal data with a transformer that operates on high frequency signals. We incorporate low-frequency signals as diffusion conditions in the transformer architecture to capture overall spatio-temporal profiles and gradual trends. To enhance the learning of each step, we design an diffusion model featuring a spatio-temporal attention module that adaptively captures interdependencies between time and space. Extensive experiments across diverse domains including traffic, economics, and environment demonstrate that our method significantly outperforms state-of-the-art baselines.
BibTeX
@inproceedings{qin2026wavedist,
  author = {Huiling Qin and Yuanxun Li and Weijia Jia},
  title = {{WaveDiST: A Wavelet Diffusion Transformer for Spatio-Temporal Estimation on Unobserved Locations}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2026},
  doi = {10.1609/aaai.v40i18.38593},
}

Loading related content...

Figure