A comprehensive survey of deep learning for time series forecasting: architectural diversity and open challenges

Jongseon Kim, Hyungjoon Kim, HyunGi Kim, Dongjun Lee, Sungroh YoonArtif. Intell. Rev 2025 · OpenReview

Abstract

Time series forecasting (TSF) is a critical task across domains like economic planning, supply chain management, and medical diagnosis. While Transformers once dominated the field due to their ability to handle long-term dependencies, a recent "renaissance" has seen simple linear models and emerging architectures—like Mamba and Diffusion models—challenge their supremacy. This survey provides a timely analysis of this architectural diversification, re-examining fundamental models and exploring new trends like foundation models. It also addresses core challenges such as channel dependency, distribution shifts, and causality, offering a systematic roadmap for both newcomers and seasoned researchers.

Introduction: The TSF Renaissance

Time series forecasting is notoriously difficult due to complex temporal dependencies, noisy data, and non-stationary distributions. For several years, the field followed a "Transformer-first" paradigm. However, the recent discovery that simple multi-layer perceptrons (MLPs) could outperform complex Transformers on long-term forecasting benchmarks sparked a significant shift toward architectural diversity.

Growth of TSF research papers across major AI conferences (2020-2024).

As shown in the figure above, the volume of TSF research has exploded in recent years, reflecting a renewed interest in exploring varied deep learning approaches beyond standard attention mechanisms. This survey aims to unify these diverse developments into a single roadmap.

Scope of the Survey

Unlike previous reviews that may focus narrowly on specific architectures, this work provides a comprehensive overview of the evolution of TSF models, including a discussion of critical open challenges and advanced solutions.

Comparison of this survey's comprehensive focus against existing literature.

Taxonomy of Modern Architectures

The survey categorizes the TSF landscape into several distinct eras and methodologies:

The Transformer Era and Its Refinements

To address the quadratic computational cost of standard Transformers, models like Informer and PatchTST introduced sparse attention and "patching." Patching, which groups time series data into local semantic units (similar to words in a sentence), has proven essential for preserving local context and drastically reducing computational complexity.

The Post-Transformer Renaissance

The resurgence of fundamental models has been a defining trend. MLP-based models (e.g., LTSF-Linear) demonstrated that the permutation-invariant nature of standard attention can sometimes destroy crucial temporal order information. Simultaneously, modern CNNs and GNNs continue to excel in local pattern extraction and spatio-temporal tasks, such as traffic network forecasting.

Emerging Frontiers: Mamba, Diffusion, and Foundation Models

  • Mamba/State Space Models (SSMs): These offer linear-time sequence modeling, providing the efficiency of RNNs with the performance of Transformers, making them ideal for extremely long sequences.
  • Diffusion Models: Increasingly used for probabilistic forecasting, these models are becoming the gold standard for capturing uncertainty.
  • Foundation Models: Leveraging massive pre-training (e.g., TimesFM, Chronos) or reprogramming text-based LLMs (e.g., Time-LLM) to achieve zero-shot forecasting capabilities across different domains.

Addressing Open Challenges

Beyond architecture, the survey identifies critical hurdles that impact forecasting performance:

  • Channel Independence (CI) vs. Dependence (CD): While multivariate models theoretically benefit from cross-variable correlations (CD), treating channels independently (CI) often yields better results. CI acts as a regularizer, making models more robust to noisy correlations and distribution shifts.
  • Distribution Shifts: Real-world data is non-stationary. The survey reviews normalization techniques designed to help models adapt when data properties change over time.
  • Frequency Domain Learning: Utilizing Fourier or Wavelet transforms allows models to capture global periodicities and filter high-frequency noise more effectively than time-domain processing alone.

Conclusion

The field of TSF has moved beyond a one-size-fits-all approach. Architectural diversity is now the norm, with different models excelling in different environments—from resource-constrained MLPs to spatio-temporal GNNs. Future work will likely involve integrating domain-specific knowledge and exploring neuro-symbolic AI to create more adaptable, generalized models for complex, real-world environments.

Summary updated

Abstract

Time series forecasting is a critical task that provides key information for decision-making across various fields, such as economic planning, supply chain management, and medical diagnosis. After the use of traditional statistical methodologies and machine learning in the past, various fundamental deep learning architectures such as MLPs, CNNs, RNNs, and GNNs have been developed and applied to solve time series forecasting problems. However, the structural limitations caused by the inductive biases of each deep learning architecture constrained their performance. Transformer models, which excel at handling long-term dependencies, have become significant architectural components for time series forecasting. However, recent research has shown that alternatives such as simple linear layers can outperform Transformers. These findings have opened up new possibilities for using diverse architectures, ranging from fundamental deep learning models to emerging architectures and hybrid approaches. In this context of exploration into various models, the architectural modeling of time series forecasting has now entered a renaissance. This survey not only provides a historical context for time series forecasting but also offers comprehensive and timely analysis of the movement toward architectural diversification. By comparing and re-examining various deep learning models, we uncover new perspectives and present the latest trends in time series forecasting, including the emergence of hybrid models, diffusion models, Mamba models, and foundation models. By focusing on the inherent characteristics of time series data, we also address open challenges that have gained attention in time series forecasting, such as channel dependency, distribution shift, causality, and feature extraction. This survey explores vital elements that can enhance forecasting performance through diverse approaches. These contributions help lower entry barriers for newcomers by providing a systematic understanding of the diverse research areas in time series forecasting (TSF), while offering seasoned researchers broader perspectives and new opportunities through in-depth exploration of TSF challenges.
BibTeX
@inproceedings{kim2025comprehensive,
  author = {Jongseon Kim and Hyungjoon Kim and HyunGi Kim and Dongjun Lee and Sungroh Yoon},
  title = {{A comprehensive survey of deep learning for time series forecasting: architectural diversity and open challenges}},
  booktitle = {Artif. Intell. Rev},
  year = {2025},
  url = {https://openreview.net/forum?id=TqR2ZaquY8},
}
Figure