The Complexity Performance Gap in Time Series Forecasting

47 concepts36 papers1 proposal

In machine learning, we often operate under the assumption that greater architectural complexity—more parameters, deeper networks, and expressive probabilistic modeling—inherently translates to superior predictive accuracy. However, across a wide variety of sequential and time-series forecasting domains, a sobering paradox has emerged: highly sophisticated deep learning architectures frequently struggle to outperform "naive" baselines.

Whether predicting retail sales, traffic speeds, or pandemic trajectories, simple heuristics—such as assuming the future will exactly mirror the most recent observation (the persistence baseline) or calculating a basic historical average—regularly match or exceed the performance of state-of-the-art neural networks. As Riccardo Ughi et al. argue in Two Steps Forward and One Behind: Rethinking Time Series Forecasting with Deep Learning, the field frequently neglects these simple non-parametric baselines, leading to the adoption of heavy architectures whose actual predictive utility is illusory. To build better models, we must first understand the mathematical and structural reasons why these trivial baselines are so difficult to beat.

The Power of System Inertia and Autocorrelation

The primary reason naive baselines are so formidable is that many real-world processes are governed by strong temporal dependence, or physical inertia. In short-horizon forecasting, the immediate past contains an overwhelming amount of mutual information about the immediate future.

Consider short-term wind forecasting. Chris Barber et al. demonstrated in Auto-Regressive HMM Inference with Incomplete Data for Short-Horizon Wind Forecasting that simple persistence models often outperform complex physical simulations because capturing the immediate inertia of the weather system is paramount for short-range accuracy. Similarly, in building control systems, Qie Hu et al. found in Building Model Identification during Regular Operation - Empirical Results and Challenges that a persistence-based heuristic assuming internal heat gains remain constant over short intervals provides a remarkably robust baseline for online learning without requiring complex real-time sensing.

When an environment is highly autocorrelated, a zero-intelligence model that simply repeats the last observation acts as an incredibly accurate anchor. Albert Bifet et al. highlighted this in Evaluation methods and decision theory for classification of streaming data with temporal dependence, showing that without benchmarking against a naive persistent baseline, researchers can easily misattribute a model's success to its complexity when it is actually underperforming compared to simple repetition. This phenomenon spans stable environments like optical networks (Somaiieh Yousefi et al. in Forecasting Lightpath Quality of Transmission and Implementing Uncertainty in the Forecast Models) and highly volatile ones like short-term financial forecasting (Aleksandr Pletnev et al. in Graph Neural Networks for Model Recommendation using Time Series Data), where persistence remains the optimal choice in up to 43% of cases.

Overfitting to Noise vs. Modeling Signal

If system inertia explains why simple baselines are strong, what explains why complex models are weak? The answer lies in the trade-off between variance and bias. High-capacity models like deep LSTMs, Transformers, and complex probabilistic architectures are designed to capture intricate, non-linear relationships. However, real-world time-series data is often dominated by transient noise, structural shifts, or data sparsity.

When evaluated on point estimation metrics like RMSE, sophisticated probabilistic models often degrade because the complexity required to map full probability distributions introduces noise and leads to overfitting. In a large-scale analysis of sales forecasting, Seungjae Jung et al. found in A Worrying Analysis of Probabilistic Time-series Models for Sales Forecasting that simple multi-layer perceptrons and linear regressions consistently outperformed complex autoregressive or state-space models. The authors noted that architectural sophistication does not inherently solve the practical challenges of noisy retail environments.

This pattern repeats across diverse domains:

Historical Averages as Robust Regularizers

When a system moves beyond immediate short-term horizons where persistence dominates, the next most effective baseline is often a historical average or frequency metric. While theory-based computational models or deep architectures attempt to simulate underlying cognitive or physical mechanisms, basic historical frequency acts as a powerful regularizer.

For instance, in behavior prediction, Chao Zhang et al. found in Theory-based habit modeling for enhancing behavior prediction in behavior change support systems that complex psychological models failed to provide immediate predictive gains over simple past-behavior averages. Similarly, for dynamic demand response in smart buildings, Saima Aman et al. demonstrated in Prediction Models for Dynamic Demand Response that averaging models using only a few days of recent data offered the optimal balance of accuracy and computational efficiency. By compressing historical patterns into a single mean or frequency, these models eliminate the variance that causes deeper architectures to fail in univariate or noisy contexts (Mingli Song et al. in A robust federated biased learning algorithm for time series forecasting).

Restoring Rigor to Time-Series Evaluation

The recurring failure of complex models to decisively beat naive baselines highlights a systemic issue in machine learning research: the lack of rigorous benchmarking. To truly validate a new architecture, it must be tested against non-parametric baselines that exploit system inertia and historical distributions.

If a deep Transformer model (Shihao Yang et al. in County augmented transformer for COVID-19 state hospitalizations prediction) or a massive multi-model ensemble (Mostafa Farrokhabadi et al. in Day-Ahead Electricity Demand Forecasting Competition: Post-COVID Paradigm) cannot outperform a flat-line projection or a historical average, its computational cost and deployment complexity are unjustified. Understanding the strength of these simple baselines allows researchers to design architectures that intentionally model what the baseline misses, rather than accidentally reinventing a worse version of a simple average.

Key Foundational Work

Auto-Regressive HMM Inference with Incomplete Data for Short-Horizon Wind Forecasting (2010) — contributed 1 concept elements to this direction.
Experiments in UNIX Command Prediction (1997) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_naive_baseline_performance_gap_da0a7e02,
  title = {naive baseline performance gap},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/naive-baseline-performance-gap/rdr_da0a7e024e8b482c97e60c1f82d2a15c},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_naive_baseline_performance_gap_da0a7e02_v1,
  title = {naive baseline performance gap},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/naive-baseline-performance-gap/rdr_da0a7e024e8b482c97e60c1f82d2a15c/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake