Abstract
Accurate forecasting of long-term time series has important applications for decision making and planning. However, it remains challenging to capture the long-term dependencies in time series data. To better extract long-term dependencies, We propose Multi Scale Dilated Convolution Network (MSDCN), a method that utilizes a shallow dilated convolution architecture to capture the period and trend characteristics of long time series. We design different convolution blocks with exponentially growing dilations and varying kernel sizes to sample time series data at different scales. Furthermore, we utilize traditional autoregressive model to capture the linear relationships within the data. To validate the effectiveness of the proposed approach, we conduct experiments on eight challenging long-term time series forecasting benchmark datasets. The experimental results show that our approach outperforms the prior state-of-the-art approaches and shows significant inference speed improvements compared to several strong baseline methods.
Introduction
Long-term time series forecasting—predicting values far into the future—is critical for fields like energy management and traffic planning. However, existing models face a "Goldilocks" problem. Transformers are powerful but suffer from high computational complexity and "permutation invariance," which can make them blind to the specific order of time steps. Standard CNNs are fast but often only capture short-term local patterns, missing the "big picture" dependencies like yearly cycles. Finally, MLP-based models are extremely fast but can be too simple to capture complex, non-linear temporal dynamics.
MSDCN aims to bridge this gap by creating a model that is deep enough to capture multi-scale patterns (weekly, monthly, yearly) but shallow enough to remain computationally efficient.
Proposed Method: MSDCN
The core of MSDCN is a shallow architecture that leverages multi-scale dilated convolutions to expand the receptive field without the overhead of deep networks.
Multi-Scale Dilated Convolutions
Unlike standard convolutions, dilated convolutions skip points in the input data. This allows the receptive field to grow exponentially without needing a massive number of parameters. The model uses two parallel modules:
- Long Convolution Module: Uses large kernels to capture broad, long-term trends.
- Short Convolution Module: Uses smaller kernels for fine-grained, short-term dependencies.
Within these modules, dilation rates grow by factors of two (e.g., $2^0+1, 2^1+1, \dots$), allowing the model to sample the time series at multiple resolutions simultaneously, as illustrated below.
Hybrid Architecture
Recognizing that many time series have strong linear components, MSDCN includes a traditional Autoregressive (AR) module. This acts like a "skip connection" that handles simple linear relationships, leaving the convolutional layers to focus on complex, non-linear patterns. For training, the model employs Huber Loss, which provides greater robustness to outliers than standard Mean Squared Error (MSE).
Experimental Evaluation
The model was evaluated on eight widely recognized long-term forecasting benchmarks, including the Electricity Transformer Temperature (ETT) datasets, Electricity consumption, Weather indicators, Traffic occupancy, and Influenza-like illness (Illness) data.
Key Results
- Superior Accuracy: MSDCN achieved the best results in 80% of the prediction tasks across the eight datasets. Compared to the popular ETSformer, MSDCN showed a 9.4% reduction in MSE and an 11.0% reduction in MAE.
- Exceptional Efficiency: On the Traffic dataset, MSDCN performed inference in 3.68 seconds, compared to 62.39 seconds for Autoformer and 1491.58 seconds for TimesNet. It approaches the speed of simple linear models while maintaining significantly higher accuracy.
Discussion and Conclusion
MSDCN demonstrates that a shallow network can outperform deep Transformers if the convolutions are designed to see enough of the past. The hybrid approach of combining non-linear CNNs with a linear AR module ensures the model doesn't waste complex parameters on simple trends.
While highly effective, the current iteration treats different variables independently, which may miss complex cross-channel correlations. Future work may involve making dilation factors adaptive to specific periodicities within the data to further refine the multi-scale representation.
Summary updated