Recursive Multi-Step Forecasting for Long-Term Dynamics

216 concepts185 papers

Predicting the future is rarely a single-step task. Whether forecasting the path of a hurricane, the spread of a virus, or the next frame in a video, the horizon of interest often extends far beyond the next immediate observation. Recursive multi-step forecasting addresses this by training a model to predict a single short-term transition and then iteratively feeding that prediction back as input to generate subsequent steps.

The Core Mechanism: Learning the Transition Function

The fundamental intuition behind recursive forecasting is that complex, long-term behavior can be decomposed into a series of local transitions. Instead of training a model to map a current state $x_t$ directly to a distant future state $x_{t+k}$, researchers train the model to learn the underlying "transition function" that maps $x_t$ to $x_{t+1}$.

As Lapedes et al. demonstrated in their foundational work on chaotic systems, this iterative approach allows a model to maintain higher stability and accuracy over long horizons than a single direct-prediction model. By recursively applying the learned transition, the model effectively simulates the dynamical system's evolution. This strategy has been applied across diverse fields, from Seo et al. predicting dam inflows to Laurenti et al. forecasting fault zone stress in earthquakes.

Managing Drift and Error Propagation

The primary challenge of recursive forecasting is "drift": small errors in the first prediction are fed back as inputs, where they can be amplified in the next step, eventually leading the model away from a plausible trajectory. Grigoryeva et al. provided a theoretical basis for this, showing that prediction error in iterated forecasting grows exponentially at a rate dictated by the system's Lyapunov exponent.

To combat this, researchers have developed several stabilizing strategies:

  • Short-Window Training: Barquero et al. found that training Transformers on very short temporal windows and using them autoregressively prevents the model from regressing to a "static mean"—a common failure mode where the model predicts an average, blurry state to minimize loss.
  • Block-wise Prediction: Rather than predicting one step at a time, Ciamarra et al. and the authors of Accelerating Simulation of Two-Phase Flows with Neural PDE Surrogates propose predicting "chunks" or bundles of future timesteps. This reduces the number of recursive iterations required, thereby limiting the opportunities for error to accumulate.
  • Nested Sub-forecasts: Lu et al. suggest generating a sequence of nested sub-forecasts of increasing length, allowing shorter-range predictions to provide a stable foundation for longer-range ones.

Latent and Semantic Projections

A powerful way to improve recursive stability is to forecast in a space that is more structured than raw data. For example, Neverova et al. found that predicting future semantic segmentation maps (labels like "car" or "road") is more effective than predicting raw pixels, as the high-level labels are more stable over time.

Similarly, Han et al. and Devarakonda et al. utilize latent representations. By iteratively generating the next hidden state or feature vector rather than the raw output, the model can capture long-term temporal dependencies without being distracted by high-frequency noise. This "belief state" approach, as explored by Yang et al., allows the model to maintain an internal representation of the future that evolves as it generates new steps.

Modern Paradigms: LLMs and Evolutionary Training

Recent work has shifted toward repurposing large-scale models for recursive tasks. Huang et al. and Xue et al. demonstrate that Large Language Models (LLMs) can be used as autoregressive forecasters by projecting numerical data into the model's embedding space. The LLM’s native "next-token prediction" workflow naturally handles the recursive feeding of outputs back into the input context.

Furthermore, the paradigm of Evolutionary Forecasting suggests a decoupling of training and inference. By training a model strictly on short-term dynamics, it can be used as a recurrent operator to generate arbitrary-length forecasts at inference time. This approach often outperforms models trained specifically for long horizons, as it avoids the gradient conflicts and optimization difficulties inherent in long-sequence training.

Key Foundational Work

How Neural Nets Work (1987) — contributed 2 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 175 papers
  2. Version 1 · 2026-10-04 · 256 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_recursive_multi_step_forecasting_ea793165,
  title = {recursive multi-step forecasting},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/recursive-multi-step-forecasting/rdr_ea79316566c745c4bf1a61f039e9a6e4},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_recursive_multi_step_forecasting_ea793165_v2,
  title = {recursive multi-step forecasting},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/recursive-multi-step-forecasting/rdr_ea79316566c745c4bf1a61f039e9a6e4/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}

See something wrong? Report a mistake