Introduction
Multivariate time series (MTS) data, such as traffic flow patterns across a city or air quality readings from a network of sensors, are fundamental to modern urban planning and environmental monitoring. However, in real-world scenarios, these datasets are frequently incomplete due to sensor malfunctions, transmission errors, or power failures. Accurate imputation—the process of filling in these missing values—is essential for any meaningful downstream analysis.
Existing deep learning approaches to imputation often face a "Goldilocks" problem regarding temporal feature learning. Recurrent Neural Networks (RNNs) tend to accumulate errors over long sequences. Attention mechanisms excel at capturing global, long-term dependencies but often overlook critical local, short-term fluctuations. Conversely, Convolutional Neural Networks (CNNs) are adept at capturing local patterns but are constrained by a fixed "receptive field," limiting their ability to see the broader temporal context.
To address these limitations, we introduce Score-CDM, a novel diffusion-based model that adaptively balances local and global temporal features through a specialized denoising backbone.
The Score-CDM Framework
Score-CDM operates within the Diffusion Probabilistic Model framework. This generative approach learns to reconstruct clean data by reversing a process that gradually adds Gaussian noise to the original signal. The core innovation of Score-CDM lies in its denoising function, which utilizes two primary modules to capture complex temporal correlations.
1. Score-weighted Convolution Module (SCM)
The SCM is designed to capture global temporal features. It generates a "score map," which functions as a globally attentive convolution kernel. Unlike standard attention mechanisms that simply weight values, the SCM uses these weights to perform a globally aware convolution across the entire time domain. This allows the model to maintain the global reach of Transformers while leveraging the feature-extraction strengths of convolutional architectures.
2. Adaptive Reception Module (ARM)
To handle local features and periodicities, the ARM utilizes a Spectral2Time Window Block (S2TWB). This module employs the Fast Fourier Transform (FFT) to transition data into the spectral domain. Based on the convolution theorem—where multiplication in the spectral domain is equivalent to convolution in the time domain—the ARM can adaptively adjust its receptive field. This enables the model to "zoom in" on local fluctuations or "zoom out" to capture broader trends more flexibly than a standard CNN with a fixed kernel size.
By integrating the global score map from the SCM with the adaptive windowing of the ARM, Score-CDM achieves a flexible receptive field capable of modeling multi-scale temporal dependencies.
Experimental Evaluation
We evaluated Score-CDM on three prominent real-world multivariate time series benchmarks:
- METR-LA: Traffic flow data from 207 sensors in Los Angeles.
- PEMS-BAY: Traffic flow data from 325 sensors in the San Francisco Bay Area.
- AQI-36: Air quality index data from 36 sensors in Beijing.
The model was tested under two challenging scenarios: Point Missing (randomly distributed missing data) and Block Missing (consecutive missing data simulating total sensor failure).
Key Results and Insights
Score-CDM consistently outperformed state-of-the-art (SOTA) diffusion models, including PriSTI and CSDI, across all datasets.
- Superior Accuracy: On the PEMS-BAY dataset, Score-CDM achieved a 9% to 12% improvement in Mean Absolute Error (MAE) over the best-performing baseline.
- Robustness to Sparsity: The model demonstrated remarkable resilience to data loss; even when 95% of the data points were missing, Score-CDM maintained high accuracy, significantly outperforming RNN and Transformer-based alternatives.
- Handling Sensor Failure: In the "block missing" scenario, Score-CDM outperformed PriSTI by over 3% on both METR-LA and AQI-36, proving its efficacy in reconstructing long gaps in data.
- Ablation Success: Internal testing confirmed that both the SCM and ARM are vital; removing either module resulted in a significant performance decline, validating the necessity of balancing global score-weighting with local spectral-windowing.
Conclusion and Future Work
Score-CDM represents a significant step forward in MTS imputation by bridging the gap between global attention and local convolution. By leveraging the spectral domain for adaptive receptive fields, it provides a more nuanced reconstruction of complex temporal data.
Future research will focus on deeper integration of graph-based spatial learning to better capture the relationships between sensors. Additionally, while the model is efficient ($O(Td + T \log T)$), further optimization of the FFT operations within the diffusion reverse process could enhance its suitability for real-time, ultra-large-scale deployments.
Summary updated