TL;DR
This paper introduces WFTNet, a novel deep learning architecture for long-term time series forecasting that effectively combines Fourier and Wavelet transforms to capture both global and local periodic patterns. By adaptively balancing these two types of frequency information using a Periodicity-Weighted Coefficient (PWC), WFTNet achieves state-of-the-art performance across various benchmark datasets.
What Problem Does This Paper Address?
Long-term time series forecasting is a critical task with applications in finance, weather prediction, and traffic flow. A major challenge in this domain is the inherent complexity of time series data, which often exhibits intricate temporal variations characterized by both global and local periodicities. While recent Transformer-based models have shown promise, they often struggle to fully utilize frequency and periodicity information, particularly failing to capture fine-grained, local frequency structures. Existing methods like TimesNet, which leverage Fourier transforms, excel at global periodicity but overlook crucial local patterns. This paper aims to bridge this gap by developing a model that can effectively exploit both global and local periodic characteristics for more accurate long-term forecasting.
Methods
The core of WFTNet is its WFTBlock, which processes 1D time series data into 2D temporal-frequency representations. This block comprises two main stages:
- Feature Extraction Stage: The input time series is fed into two parallel branches:
- Fourier Transform Branch: This branch uses the Fast Fourier Transform (FFT) to capture global periodic patterns. It identifies the top-k frequencies with the highest amplitudes and then segments the time series into blocks corresponding to these periods, creating a 2D frequency map.
- Wavelet Transform Branch: This branch employs the Continuous Wavelet Transform (CWT), specifically using the Morlet wavelet. CWT is adept at capturing local period structures and provides superior time-frequency localization, allowing for the analysis of non-stationary signals where frequencies change over time. This results in a time-frequency map that offers better local detail.
- Feature Fusion Stage: The outputs from both branches are combined.
- The Fourier branch's output is weighted by its corresponding frequency amplitudes using a softmax normalization, creating an importance-adjusted composite frequency representation.
- The wavelet branch's output is processed by a specialized strip-shaped convolutional kernel to reduce the scale dimension.
- Crucially, a Periodicity-Weighted Coefficient (PWC) is introduced to adaptively balance the contributions from the Fourier and wavelet branches. The PWC quantifies the inherent periodicity of the signal, allowing the model to dynamically adjust the weight given to global versus local frequency features. This is achieved by calculating the average ratio of maximum to total energy across the first 'm' frequencies.
WFTNet utilizes a standard encoder-decoder architecture, where the WFTBlocks are integrated within the encoder to manage input normalization, embedding, and output projection.
Datasets
The paper evaluates WFTNet on seven benchmark time series datasets:
- Electricity Transformer Temperature (ETT) with its four sub-datasets: ETTh1, ETTh2, ETTm1, ETTm2.
- Traffic
- ECL
- Weather
For each dataset, the data is split with 70% for training, 20% for testing, and 10% for validation.
Key Insights and Ideas
- Complementary Nature of Fourier and Wavelet Transforms: The paper highlights that Fourier transform is excellent for global periodicity, while Wavelet transform excels at local periodicity. Combining them offers a more comprehensive view of the signal's temporal-frequency characteristics.
- The Importance of Local Periodicity: Many existing models overlook local periodic patterns, which are crucial for accurate forecasting, especially in non-stationary time series. WFTNet's Wavelet branch directly addresses this.
- Adaptive Balancing is Key: Simply using Fourier or Wavelet alone is insufficient. The introduction of the Periodicity-Weighted Coefficient (PWC) is a significant innovation, allowing the model to dynamically adjust its focus based on the inherent periodicity of the specific time series, leading to improved performance.
- PWC's Adaptability: The PWC's ability to emphasize global periodicity (value near 1) or local behavior (value near 0) makes WFTNet versatile across different types of time series, as demonstrated by its performance on ECL (strong periodicity) and ETTh2 (less periodicity).
Key Results
WFTNet consistently outperforms state-of-the-art baseline models, including TimesNet, ETSformer, DLinear, FEDformer, and Autoformer, across various output sequence lengths and datasets. For instance, on the ETT dataset with a 96-step output sequence, WFTNet achieves an MSE of 0.164 and MAE of 0.267, which are the best reported results. The ablation study in Table 2 further validates the effectiveness of the PWC, showing that WFTNet surpasses both its "Fourier-Only" and "Wavelet-Only" variants on ECL and ETTh2 datasets.
Limitations and Future Work
- Computational Cost: While not explicitly detailed as a limitation, the use of both Fourier and Wavelet transforms, along with convolutional structures, might imply a higher computational cost compared to simpler models.
- Hyperparameter Tuning: The PWC introduces a hyperparameter 'a' and the number of frequencies 'm' to consider. Optimal tuning of these parameters for diverse datasets might require further investigation.
- Future Work: The paper suggests that WFTNet's architecture can be extended to handle multivariate time series forecasting and potentially applied to other domains requiring temporal-frequency analysis. The adaptive nature of the PWC also opens avenues for exploring more sophisticated adaptive mechanisms.
Summary updated