Score-CDM: Score-Weighted Convolutional Diffusion Model for Multivariate Time Series Imputation

Shunyang Zhang, Senzhang Wang, Hao Miao, Hao Chen, Changjun Fan, Jian Zhang

Abstract

Multivariate Time Series (MTS) data often suffers from missing values due to sensor failures or communication issues, hindering downstream analysis. We present Score-CDM, a novel diffusion-based generative model designed for high-fidelity MTS imputation. Score-CDM integrates a Score-weighted Convolution Module (SCM) and an Adaptive Reception Module (ARM) to capture both local fluctuations and long-term global trends. By utilizing spectral-domain convolutions via Fast Fourier Transforms (FFT), the model achieves a flexible receptive field with $O(T \log T)$ complexity. Our evaluation on real-world benchmarks including METR-LA, PEMS-BAY, and AQI-36 shows that Score-CDM consistently outperforms existing state-of-the-art methods in both point-missing and block-missing scenarios.

Introduction

Multivariate Time Series (MTS) data is ubiquitous in domains like urban traffic monitoring and environmental sensing. However, real-world data is frequently incomplete. Effective imputation is challenging because it requires modeling complex temporal dependencies: local features (e.g., sudden spikes) and global features (e.g., seasonal cycles). While CNNs excel at local patterns and Attention mechanisms capture global context, existing hybrids often fail to adaptively balance these features. Score-CDM addresses this by using a "score map" to weight convolutional kernels, creating a globally attentive convolution.

Methodology

Score-CDM serves as the denoising engine within a diffusion framework. It comprises two primary innovations:

Score-weighted Convolution Module (SCM)

The SCM learns a global "score map" that assigns weights to every time step. This map functions as a data-driven convolution kernel, identifying the most relevant segments of the time series for a given imputation task.

Adaptive Reception Module (ARM)

The ARM utilizes a Spectral2Time Window Block (S2TWB). By transitioning into the frequency domain using FFT, the model can perform complex convolutions efficiently. This allows for a flexible receptive field that naturally captures periodicities of varying lengths.

The architecture of Score-CDM, highlighting the SCM and ARM modules.

The synergy between the global score map and the flexible spectral windows enables the model to handle both short-term jitters and long-term cycles simultaneously.

The transition between Time and Frequency domains within the S2TWB, enabling a flexible receptive field.

Datasets and Evaluation Scenarios

To rigorously test the model's capabilities, the researchers utilized three real-world Multivariate Time Series benchmarks:

  • METR-LA: Traffic flow data from 207 sensors in Los Angeles.
  • PEMS-BAY: Traffic flow data from 325 sensors in the San Francisco Bay Area.
  • AQI-36: Air quality index data from 36 sensors in Beijing.

Two distinct missing-data scenarios were simulated:

  1. Point Missing: Randomly scattered gaps across the dataset, testing the model's ability to handle sporadic data loss.
  2. Block Missing: Continuous segments of missing data, simulating real-world sensor failures that last for several hours. This is significantly more difficult as it requires the model to rely heavily on long-term global trends.

Experimental Results

Score-CDM consistently outperformed all baselines, including popular models like CSDI, PriSTI, and TimesNet.

Performance Highlights

  • Accuracy: On the AQI-36 dataset, Score-CDM improved Mean Absolute Error (MAE) by 3% to 5% over previous leaders.
  • Robustness: In the challenging "Block Missing" scenario on PEMS-BAY, it outperformed the best baseline, PriSTI, by over 12%.
  • Efficiency: The FFT-based approach maintains $O(T \log T)$ complexity, making it more scalable than standard $O(T^2)$ attention mechanisms for long sequences.

Conclusion

Score-CDM represents a significant step forward in MTS imputation by effectively bridging the gap between local and global feature extraction. While future work may further optimize spatial-temporal "channel mixing," the current model provides a robust and efficient solution for recovering missing data in complex time series.

Summary updated

BibTeX
@inproceedings{zhang2024score,
  author = {Shunyang Zhang and Senzhang Wang and Hao Miao and Hao Chen and Changjun Fan and Jian Zhang},
  title = {{Score-CDM: Score-Weighted Convolutional Diffusion Model for Multivariate Time Series Imputation}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
  url = {https://openreview.net/forum?id=zjn4lTsiU6},
}
Figure