Introduction
Long-Term Time Series Forecasting (LTSF) is a critical component of strategic planning across various sectors, including energy, finance, and meteorology. Accurate predictions often necessitate a "long lookback window" to capture historical patterns. However, increasing the lookback window introduces a significant challenge: the inclusion of temporal noise, which can lead to overfitting in complex models.
The HADL framework (Haar, DCT, Low-rank) addresses this trade-off by providing a lightweight, noise-resilient architecture. It challenges the prevailing trend of using massive Transformer-based models, demonstrating that classical signal processing combined with efficient linear layers can achieve state-of-the-art performance with a fraction of the parameters.
Methodology
The HADL framework utilizes a three-stage pipeline designed to distill underlying signals from noisy temporal data:
- Haar Wavelet Decomposition: The input series undergoes a Discrete Wavelet Transform (DWT). By retaining approximation coefficients (the trend) and discarding detail coefficients (high-frequency noise), the model achieves an initial 2x compression and denoising.
- Discrete Cosine Transform (DCT): The denoised data is projected into the frequency domain. The DCT facilitates "energy compaction," concentrating the most significant long-term patterns into a small number of coefficients.
- Low-Rank Linear Prediction Layer: Instead of a dense weight matrix, HADL employs a low-rank approximation ($W \approx PQ$). This bottleneck structure acts as a regularizer, preventing the model from memorizing residual noise and significantly reducing the parameter count.
A key innovation in HADL is the omission of the inverse transform; the model learns to map frequency-domain features directly to time-domain forecasts, enhancing computational efficiency.
Experimental Evaluation
The authors evaluated HADL on standard LTSF benchmarks, including the Electricity Transformer Temperature (ETT) datasets, Traffic, Weather, and Electricity consumption data. Performance was compared against leading Transformer, CNN, and MLP-based architectures.
Accuracy and Efficiency
Despite its minimal parameter footprint—often requiring only 17–50K parameters compared to the millions used by models like PatchTST—HADL achieved top-tier Mean Squared Error (MSE) results. It consistently outperformed or matched much larger models across the ETT datasets.
Noise Resilience
To quantify robustness, the authors introduced the Noise-Resilience Relative (NRR) metric and measured Mean Absolute Variability (MAV) under injected Gaussian noise. HADL exhibited remarkable stability, maintaining consistent performance in volatile environments where Transformer-based models showed sharp degradation.
Discussion and Conclusion
The HADL framework demonstrates that "bigger" is not always "better" in time series forecasting. By leveraging classical signal processing as a "pre-digestion" step, the model effectively filters noise before it reaches the learning layers. The use of low-rank matrices further ensures that the model remains focused on salient features rather than high-frequency fluctuations.
While HADL excels in efficiency and noise resilience, it shows some sensitivity to lookback window sizes and slightly lower performance on extremely high-dimensional datasets like Traffic. Future work may explore multi-level decompositions or adaptive coefficient selection to further refine its ability to capture fine-grained information without sacrificing its lightweight nature.
Summary updated