Introduction
Long-term time series forecasting is a cornerstone of modern infrastructure, powering applications from power grid management to weather prediction. However, real-world data is notoriously complex—it is often non-stationary and contains critical patterns hidden across disparate scales. Traditional Transformer-based models frequently struggle with "information masking," where high-dimensional embeddings obscure multivariate correlations, leading to "attention collapse."
Presented at AISTATS 2025, HAR-former (Hybrid Transformer with an Adaptive Time-Frequency Representation Matrix) introduces an architectural shift to address these challenges. By combining the stability of Multi-Layer Perceptrons (MLPs) with the relational power of Transformers, HAR-former bridges the gap between time and frequency domains, capturing both long-range dependencies and localized, high-frequency variations.
Methodology
The HAR-former architecture is built upon three primary pillars designed to handle the nuances of complex time series data.
1. Series Decomposition
The model begins by decomposing the input signal into two distinct components: a Trend component, representing slow, stable movements, and a Seasonal component, capturing repeating fluctuations.
2. The Hybrid Engine
HAR-former employs a "division of labor" strategy to optimize processing for each component:
- Trend Branch: Utilizes a 3-layer MLP combined with Reversible Instance Normalization (RevIN). Given the relative stability of trends, linear-style models prove more robust and less susceptible to noise than complex attention mechanisms in this context.
- Seasonal Branch: Employs a Transformer architecture equipped with a novel Adaptive Representation Matrix (AR-Matrix) to handle complex seasonal relationships.
3. The AR-Matrix
The AR-Matrix is the core innovation of HAR-former. It utilizes a Haar wavelet transform to decompose the seasonal signal into approximation coefficients ($a_i$) for low-frequency patterns and detail coefficients ($d_i$) for high-frequency variations.
These coefficients are concatenated into a matrix and processed via a multi-head attention mechanism. Unlike Fourier-based methods that lose temporal localization, the Haar wavelet allows the model to identify when specific frequency events occur. This enables the AR-Matrix to effectively capture localized patterns. Finally, an Inverse Haar Wavelet Transform (IHWT) is used to reconstruct the predicted signal.
Experimental Results
The authors evaluated HAR-former against state-of-the-art (SOTA) models across eight benchmark datasets, including ETT, Weather, Electricity Consuming Load (ECL), Solar-Energy, Exchange, and Traffic.
Performance and Efficiency
HAR-former consistently achieved the lowest Mean Squared Error (MSE) and Mean Absolute Error (MAE) across all benchmarks, outperforming top-tier models such as PatchTST and iTransformer. Notably, on the Exchange dataset, the integration of the AR-Matrix improved MSE by 11.31%. Furthermore, the model demonstrated significant efficiency gains; on the high-dimensional Traffic dataset (862 variables), HAR-former exhibited lower memory consumption and faster training speeds compared to standard Transformers.
Lookback Resilience
A common failure mode for Transformers in forecasting is performance degradation when provided with excessively long historical data, often due to the accumulation of noise. HAR-former exhibits unique "lookback resilience," where performance actually improves as the lookback window increases. This suggests the model is superior at filtering relevant signals from historical clutter.
Conclusion
HAR-former represents a significant advancement in long-term forecasting by effectively integrating time and frequency domain information. Its hybrid approach—leveraging MLPs for trend stability and wavelet-based Transformers for seasonal complexity—provides a robust solution to the limitations of traditional attention mechanisms. The model's efficiency and resilience to long lookback windows make it a promising candidate for large-scale industrial deployment.
Reference: Zheng, K., Long, Z., & Wang, S. (2025). HAR-former: Hybrid Transformer with an Adaptive Time-Frequency Representation Matrix for Long-Term Series Forecasting. Proceedings of the 28th International Conference on Artificial Intelligence and Statistics (AISTATS).
Summary updated