Error Accumulation in Recursive Multi-Step Forecasting

41 concepts37 papers1 proposal

When predicting the future, the most intuitive path is often to take one step at a time. If you want to forecast a time series across a long horizon—such as predicting network traffic or weather patterns weeks into the future—you can train a model to predict tomorrow's value, append that prediction to your history, and use it as an input to predict the day after tomorrow. This is known as the recursive (or iterative) forecasting strategy.

While conceptually straightforward, this approach harbors a fundamental flaw: recursive error accumulation. As Bontempi et al. and Lim et al. document, small inaccuracies in early steps act as noise injected directly back into the model's input space. Over extended horizons, these errors compound, causing the model's trajectories to drift drastically away from reality.

The Mechanics of the Snowball Effect

To understand why this drift happens so aggressively, consider a standard autoregressive model $f$ that predicts the next state $x_{t+1}$ from the current state $x_t$:

$$\hat{x}_{t+1} = f(x_t)$$

In the first step, the model receives a clean, ground-truth input $x_t$ and produces a prediction $\hat{x}_{t+1} = x_{t+1} + \epsilon_1$, where $\epsilon_1$ is a minor prediction error.

In the next step, to predict $x_{t+2}$, the model no longer evaluates a true state. Instead, it evaluates its own noisy prediction:

$$\hat{x}_{t+2} = f(\hat{x}_{t+1}) = f(x_{t+1} + \epsilon_1)$$

Because $f$ is typically a non-linear function (such as a deep neural network), it does not treat $\epsilon_1$ linearly. If the model has only been trained on clean ground-truth data, the input $x_{t+1} + \epsilon_1$ represents an out-of-distribution sample. The model's response to this unfamiliar, noisy input generates a second error $\epsilon_2$, which is often significantly larger than $\epsilon_1$.

By step $H$, the input is a composite of historical errors fed back into the system. This creates what Salloom et al. describe as a "snowball effect," where early inaccuracies are amplified through a recursive feedback loop, rapidly destroying long-term predictive skill.

The Intractable Explosion of Uncertainty

The problem intensifies when shifting from point predictions to probabilistic forecasting. To safely navigate future horizons, a model must quantify its own uncertainty. However, as Rügamer et al. point out, recursive prediction requires a mechanism to propagate the full probability distribution forward.

If a model only feeds its mean point prediction back into the loop, it falsely assumes absolute certainty about intermediate steps, severely underestimating future risk. Conversely, if it attempts to pass the entire distribution through the model, the non-linear transformations inherent in neural networks quickly make the resulting probability distributions analytically intractable and impossible to bound formally (Wicker et al.; Bevanda et al.). The variance expands rapidly, rendering long-term distributions virtually useless for decision-making.

The Direct vs. Recursive Trade-off

To bypass this feedback loop entirely, many practitioners turn to direct forecasting. Instead of stepping through time sequentially, a direct model maps historical context to the entire future horizon in a single forward pass:

$$\hat{x}_{t+1}, \dots, \hat{x}_{t+H} = g(x_t)$$

As Gómez-González et al. demonstrate, decoding the entire future path at once completely eliminates the compounding feedback loop.

However, direct forecasting introduces its own severe pathologies. As Ma et al. observe, training a model to predict long horizons directly forces it to resolve massive gradient conflicts and optimization instabilities. A model trying to optimize for day 1 and day 56 simultaneously often struggles to converge effectively.

Surprisingly, the authors show that the local optimization stability of a recursive model trained on short horizons can sometimes outweigh the risks of feedback loops. This creates a fundamental tension between recursive error accumulation and direct optimization difficulty.

Breaking the Loop: Modern Mitigations

Recognizing this trade-off, recent research focuses on architectures that retain the efficiency of short-horizon training while neutralizing error propagation.

For instance, Baron et al. introduce a copula-based method for time series foundation models that generates correlated sample paths in a single forward pass, bypassing autoregressive sampling while preserving trajectory consistency. Other approaches integrate classical control theory directly into the neural architecture; Salloom et al. utilize a Proportional-Integral-Derivative (PID) booster to dynamically correct the drift introduced by iterative feedback loops.

By treating recursive error not just as a statistical quirk, but as an out-of-distribution and uncertainty propagation problem, modern forecasting frameworks are moving away from naive autoregressive rollouts toward strategies that explicitly bound, correct, or bypass the feedback loop.

Key Foundational Work

A review and comparison of strategies for multi-step ahead time series\n forecasting based on the NN5 forecasting competition (2011) — contributed 2 concept elements to this direction.
Intelligent Prediction of Fan Rotation Stall in Power Plants Based on Pressure Sensor Data Measured In-Situ (2014) — contributed 1 concept elements to this direction.
Fast Algorithm for Non-Stationary Gaussian Process Prediction (2014) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_recursive_error_accumulation_iterative_e_03f1d6de,
  title = {recursive error accumulation (iterative error accumulation)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/recursive-error-accumulation-iterative-error-accumulation/rdr_03f1d6de9a464a989d1728cf54223e5e},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_recursive_error_accumulation_iterative_e_03f1d6de_v1,
  title = {recursive error accumulation (iterative error accumulation)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/recursive-error-accumulation-iterative-error-accumulation/rdr_03f1d6de9a464a989d1728cf54223e5e/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake