Introduction
Diffusion models have emerged as a powerful class of generative models, particularly in the domain of image synthesis. However, their application to time series forecasting is complicated by the inherent nature of the diffusion process. Standard diffusion models typically add independent, identically distributed (i.i.d.) Gaussian noise to data until it reaches a state of pure chaos, subsequently learning to reverse this process. In time series, the most critical information is often the temporal correlation—the relationship between successive observations. The addition of i.i.d. noise acts as a "shredder" for these relationships, particularly affecting low-frequency trend components. This loss of structural priors makes it difficult for models to reconstruct the original temporal "rhythm" during denoising, leading to directional biases and poor long-term predictive performance.
Methodology
The proposed Temporal Correlation-Empowered Diffusion Model (TCDM) addresses these limitations through a "divide-and-conquer" approach, utilizing three core components to preserve temporal structure.
Trend-Residual Decomposition
TCDM decomposes the input time series into a smooth, long-term trend and high-frequency residuals. A frequency-domain MLP, acting as the Base Model, captures the trend component. This allows the diffusion model to focus exclusively on the more complex residual patterns, which are often harder to model using traditional methods.
Maintaining Temporal Correlation (MTC) Module
The MTC module is the central innovation of TCDM. Instead of adding completely random noise at each diffusion step, TCDM introduces Temporally Correlated Noise. This noise is generated via an autoregressive process, ensuring that the noise added to a specific point $i$ is dependent on the noise added to point $i-1$. This mechanism forces the model to maintain local temporal correlations even in highly noised states, providing a structural constraint that guides the reverse denoising process.
Redesigned Initial Module (RIM)
Standard diffusion models typically begin the reverse process from pure Gaussian noise. TCDM introduces the Redesigned Initial Module (RIM), which truncates this process to preserve more of the natural data structure. RIM starts the reverse process from a state that is not fully "shredded." To bridge the gap between pure noise and this structured starting point, a GAN-based generator is employed to perform the transition in a single step, improving both efficiency and accuracy.
Experimental Results
The TCDM framework was evaluated on nine standard multivariate time series benchmark datasets across various domains, including energy (ETTh1, ETTm1, Electricity), environment (Weather, Wind), and finance/transport (Exchange, Traffic).
Forecasting Accuracy
TCDM achieved state-of-the-art results, recording the lowest Mean Squared Error (MSE) on six out of the nine datasets. The model attained an average rank of 1.333, significantly outperforming the next best baseline, TimeDiff, which had an average rank of 3.111.
Temporal Pattern Fidelity
To measure the model's ability to capture temporal dependencies, the authors utilized the Normalized Partial Autocorrelation Difference Index (N-PADI). TCDM demonstrated a significant advantage; for instance, on the ETTh1 dataset, TCDM achieved an N-PADI of 0.559, whereas TimeDiff scored 60.057. This indicates that TCDM is far more effective at preserving the underlying temporal "rhythm" of the data.
Discussion and Conclusion
TCDM demonstrates that by engineering the noise process to be autoregressive, diffusion models can be made to respect the inherent sequential nature of time series data. The combination of frequency-domain trend modeling and structured noise generation allows TCDM to overcome the "identity crisis" faced by standard diffusion models in forecasting tasks.
While the model shows significant promise, future work will address potential limitations, such as the sensitivity of the moving-average decomposition to extremely strong periodicities and the training complexity introduced by the GAN-based RIM module. Nonetheless, TCDM represents a major step forward in applying generative diffusion models to complex time series forecasting.
Summary updated