TL;DR
This paper introduces FDNet, a novel time series forecasting network that abandons the conventional approach of learning universal features from input sequences. Instead, FDNet focuses on extracting fine-grained local features and employs a "focal decomposition" strategy to efficiently handle long input sequences, achieving state-of-the-art performance and robustness on various benchmarks.
What Problem Does This Paper Address?
Traditional deep time series forecasting models often rely on extracting "universal feature maps" from input sequences, using mechanisms like attention or dilated convolutions to capture correlations between all input elements. However, the authors argue that this approach is not always necessary and can even be detrimental, especially for real-world time series that exhibit "distribution shift" – meaning their dynamics change over time. These universal feature extractors can get stuck in local optima and lead to performance turbulence. Furthermore, existing models struggle with the "Long Sequence Time series Input (LSTI)" problem, where excessively long input sequences can lead to parameter explosion and overfitting, even for advanced models like Transformers. The paper aims to develop a forecasting method that is efficient, robust, and practical, particularly for challenging real-world scenarios with non-stationary data and long input sequences.
Methods
FDNet introduces two core innovations:
- Decomposed Forecasting Formula: Instead of relying on universal feature extraction (ICOMs), FDNet decomposes the forecasting process. It focuses solely on extracting local, fine-grained feature maps from the input sequence. This is achieved using basic linear projection layers and 2D convolutional layers that process temporal and variate dimensions separately, while sharing weights. This approach avoids the need for complex inductive biases about time series properties, making it more general.
- Focal Input Sequence Decomposition: To address the LSTI problem, FDNet employs a novel "focal input sequence decomposition" method. This strategy divides the input sequence into several sub-sequences, with those closer to the prediction window being shorter and having more feature extraction layers. Conversely, sub-sequences further away are longer but have shallower feature extraction. This allows the model to efficiently process long sequences without a significant increase in parameters or performance degradation, and it helps mitigate overfitting.
Datasets
The paper evaluates FDNet on six real-world benchmark datasets:
- ETTh1 and ETTm2: Electricity transformer temperature datasets.
- ECL: Electricity consumption load dataset.
- Traffic: Road occupation rates in San Francisco Bay area freeways.
- Exchange: Daily exchange rates from eight countries.
- Weather: Meteorological indicators from Germany.
The authors specifically chose these datasets to represent diverse characteristics of real-world time series, including non-stationarity and varying frequencies. They also mention abandoning the ILI (Influenza-like Illness) dataset due to its small size and limited instances, which would not be suitable for testing LSTI capabilities.
Key Insights and Ideas
- Universal Features Aren't Always Necessary (or Beneficial): The paper strongly argues that relying on "Input Correlation Oriented Mechanisms" (ICOMs) to extract universal features from input sequences can be detrimental, especially for non-stationary time series exhibiting distribution shifts. FDNet demonstrates that focusing on local features can lead to more robust and generalizable forecasting.
- Focal Decomposition Solves LSTI Efficiently: The proposed focal input sequence decomposition method is a clever way to handle long input sequences. By dynamically adjusting the depth and length of sub-sequences based on their temporal proximity to the prediction target, FDNet avoids the parameter explosion and overfitting issues that plague other models.
- Simplicity Can Be Powerful: FDNet's architecture is relatively simple, primarily composed of linear projection layers and convolutional layers. This simplicity, combined with the novel decomposition strategy, leads to impressive performance and efficiency.
- Robustness to Distribution Shift: By abandoning global feature extraction and focusing on local patterns, FDNet is inherently more resilient to the distribution shift problem common in real-world time series.
Key Results
- State-of-the-Art Performance: FDNet achieved competitive forecasting performance on six benchmark datasets, outperforming thirteen SOTA baselines.
- Significant MSE Reduction: On average, FDNet reduced prediction Mean Squared Error (MSE) by 38.4% compared to other baselines.
- Efficiency and Robustness: FDNet demonstrated superior efficiency (lower GPU memory, ATPE, TIT) and robustness (lower standard deviations in errors) compared to many baselines, especially when handling long input sequences.
- LSTI Handling: FDNet showed significantly better performance and robustness than both "complicated" (Transformer/CNN-based) and "simple" (linear projection-based) baselines when dealing with long input sequences.
- Ablation Studies: Experiments confirmed the effectiveness of the focal decomposition method and the proposed 2D convolutional layers for handling inter-relationships between variates.
Limitations and Future Work
- Short Input Sequence Deficiency: The paper acknowledges that FDNet's performance can be worse than some models with relatively short input sequences. This is attributed to its reliance on longer sequences for feature extraction, which is a trade-off for its LSTI handling capabilities.
- Exchange Dataset Performance: While FDNet generally performs well, its performance on the Exchange dataset (a smaller dataset with daily frequency) shows a performance drop with longer input sequences, though the drop is less severe than for other baselines. The authors suggest this is due to the dataset's size and frequency not supporting long-term dependencies.
- Future Work: The paper implicitly suggests that further exploration of the focal decomposition method's adaptability to different forecasting formulas and ICOMs (as demonstrated with FUNet) could be a fruitful direction. The authors also highlight the generalizability and robustness of their approach, implying its potential for broader applications.
Summary updated