Abstract
Time series forecasting (TSF) is a critical task across domains like economic planning, supply chain management, and medical diagnosis. While Transformers once dominated the field due to their ability to handle long-term dependencies, a recent "renaissance" has seen simple linear models and emerging architectures—like Mamba and Diffusion models—challenge their supremacy. This survey provides a timely analysis of this architectural diversification, re-examining fundamental models and exploring new trends like foundation models. It also addresses core challenges such as channel dependency, distribution shifts, and causality, offering a systematic roadmap for both newcomers and seasoned researchers.
Introduction: The TSF Renaissance
Time series forecasting is notoriously difficult due to complex temporal dependencies, noisy data, and non-stationary distributions. For several years, the field followed a "Transformer-first" paradigm. However, the recent discovery that simple multi-layer perceptrons (MLPs) could outperform complex Transformers on long-term forecasting benchmarks sparked a significant shift toward architectural diversity.
As shown in the figure above, the volume of TSF research has exploded in recent years, reflecting a renewed interest in exploring varied deep learning approaches beyond standard attention mechanisms. This survey aims to unify these diverse developments into a single roadmap.
Scope of the Survey
Unlike previous reviews that may focus narrowly on specific architectures, this work provides a comprehensive overview of the evolution of TSF models, including a discussion of critical open challenges and advanced solutions.
Taxonomy of Modern Architectures
The survey categorizes the TSF landscape into several distinct eras and methodologies:
The Transformer Era and Its Refinements
To address the quadratic computational cost of standard Transformers, models like Informer and PatchTST introduced sparse attention and "patching." Patching, which groups time series data into local semantic units (similar to words in a sentence), has proven essential for preserving local context and drastically reducing computational complexity.
The Post-Transformer Renaissance
The resurgence of fundamental models has been a defining trend. MLP-based models (e.g., LTSF-Linear) demonstrated that the permutation-invariant nature of standard attention can sometimes destroy crucial temporal order information. Simultaneously, modern CNNs and GNNs continue to excel in local pattern extraction and spatio-temporal tasks, such as traffic network forecasting.
Emerging Frontiers: Mamba, Diffusion, and Foundation Models
- Mamba/State Space Models (SSMs): These offer linear-time sequence modeling, providing the efficiency of RNNs with the performance of Transformers, making them ideal for extremely long sequences.
- Diffusion Models: Increasingly used for probabilistic forecasting, these models are becoming the gold standard for capturing uncertainty.
- Foundation Models: Leveraging massive pre-training (e.g., TimesFM, Chronos) or reprogramming text-based LLMs (e.g., Time-LLM) to achieve zero-shot forecasting capabilities across different domains.
Addressing Open Challenges
Beyond architecture, the survey identifies critical hurdles that impact forecasting performance:
- Channel Independence (CI) vs. Dependence (CD): While multivariate models theoretically benefit from cross-variable correlations (CD), treating channels independently (CI) often yields better results. CI acts as a regularizer, making models more robust to noisy correlations and distribution shifts.
- Distribution Shifts: Real-world data is non-stationary. The survey reviews normalization techniques designed to help models adapt when data properties change over time.
- Frequency Domain Learning: Utilizing Fourier or Wavelet transforms allows models to capture global periodicities and filter high-frequency noise more effectively than time-domain processing alone.
Conclusion
The field of TSF has moved beyond a one-size-fits-all approach. Architectural diversity is now the norm, with different models excelling in different environments—from resource-constrained MLPs to spatio-temporal GNNs. Future work will likely involve integrating domain-specific knowledge and exploring neuro-symbolic AI to create more adaptable, generalized models for complex, real-world environments.
Summary updated