When building neural networks to simulate physical systems, plan robotic trajectories, or forecast complex time series, a common paradigm is the autoregressive rollout. In this setup, a model predicts the next state of a system given its current state, and then feeds its own prediction back as the input for the subsequent timestep:
$$\hat{s}_{t+1} = f(\hat{s}_t, a_t)$$
While this sequential execution allows models to simulate arbitrary horizons, it suffers from a fundamental flaw: compounding prediction error. Over long horizons, tiny inaccuracies at individual timesteps accumulate, causing the model's internal simulation to drift away from physical reality, hallucinate impossible states, or diverge entirely.
Why Errors Compound: The Feedback Loop of Drift
To understand why autoregressive rollouts fail, we must look at the distribution of inputs the model encounters during training versus inference.
During standard training, models are typically optimized using teacher forcing, where the ground-truth state $s_t$ is always provided as the input to predict $s_{t+1}$. The model learns the mapping under the assumption that its inputs are clean, physically valid states drawn directly from the true data distribution.
Training (Teacher Forcing):
[True State s_0] ---> [ Model ] ---> Predicts \hat{s}_1 (compared against True s_1)
[True State s_1] ---> [ Model ] ---> Predicts \hat{s}_2 (compared against True s_2)
Inference (Autoregressive Rollout):
[True State s_0] ---> [ Model ] ---> Predicts \hat{s}_1
|
v
[ Model ] <--- [Feedback \hat{s}_1 + \epsilon_1] ---> Predicts \hat{s}_2
|
v
[ Model ] <--- [Feedback \hat{s}_2 + \epsilon_2] ---> Catastrophic Drift
During autonomous rollout, however, the ground truth is unavailable. If the model makes a minor error $\epsilon$ at step $t$, the input for step $t+1$ becomes $\hat{s}_t = s_t + \epsilon$. This small perturbation pushes the input slightly out-of-distribution (OOD). Because the model was never trained to correct for its own imperfections, it evaluates this OOD input suboptimally, producing an even larger error at the next step.
As identified early on by Príncipe et al. in their work on chaotic systems, Dynamic Modelling of Chaotic Time Series with Neural Networks, this sensitivity causes autonomous trajectories to explode or collapse. This phenomenon spans domains:
- Physical Simulators: As Sanchez-Gonzalez et al. show in Graph Networks as Learnable Physics Engines for Inference and Control, graph neural networks simulating interacting objects accumulate drift, degrading long-term control.
- World Models for RL: In Planning in Dynamic Environments with Conditional Autoregressive Models, Hansen et al. demonstrate that sequential rollouts become less reliable over long horizons, as accumulated hallucinations diverge from physical reality.
- Discrete & Textual Domains: Even in discrete spaces, Wang et al. demonstrate in Can Language Models Serve as Text-Based World Simulators? that probabilistic language models suffer from exponential error accumulation, rendering simulated world states logically inconsistent within a few steps.
To combat this instability, researchers have developed four primary strategies: modifying the training distribution, bypassing sequential rollouts entirely, embedding continuous physical invariants, and utilizing hybrid grounding.
Strategy 1: Exposing the Model to Its Own Errors During Training
If the root cause of compounding error is that the model encounters out-of-distribution inputs during rollouts, the most direct solution is to alter the training process so the model learns to correct its own drift.
The Pushforward Trick and Temporal Bundling
Instead of optimizing exclusively for single-step predictions, the authors of Message Passing Neural PDE Solvers, Brandstetter et al., introduced the pushforward training trick. During training, the model performs a multi-step rollout using its own previous, imperfect outputs. The loss function is then computed across this entire unrolled sequence. By backpropagating through its own errors, the network explicitly learns how to steer drifting trajectories back toward the true manifold.
Block Teacher Forcing
In the context of transformer-based world models, Dedieu et al. implement block teacher forcing in Improving Transformer World Models for Data-Efficient RL. Rather than providing ground-truth tokens at every single step or completely cutting off teacher forcing, the model alternates between periods of autonomous generation and ground-truth grounding during training, stabilizing long-term imagination trajectories.
Global Feature Learning via Masking
Rather than adjusting the sequence length, Garnier approaches the problem via representation learning in MeshMask: Physics-Based Simulations with Masked Graph Neural Networks. By masking up to 40% of the spatial mesh during pre-training, the model is prevented from simply memorizing local, short-term step transitions. Instead, it is forced to capture the global underlying dynamics, yielding a 60% improvement in long-term simulation stability.
Strategy 2: Bypassing Sequential Rollouts via Generative Trajectories
An alternative paradigm eliminates the step-by-step autoregressive loop entirely. If a model generates an entire multi-step trajectory or distribution simultaneously, there are no intermediate sequential steps for errors to feed back into.
Autoregressive (Error Compounds):
s_0 ---> [Step] ---> \hat{s}_1 ---> [Step] ---> \hat{s}_2 ---> [Step] ---> \hat{s}_3 (Diverged)
Trajectory Diffusion (Global Consistency):
[ Noise Trajectory ] ---> [ Denoising U-Net ] ---> [ Consistent Plan: s_0, s_1, s_2, s_3 ]
Trajectory Diffusion
In Planning with Diffusion for Flexible Behavior Synthesis, Janner et al. treat planning not as a sequence of forward predictions, but as a global trajectory generation problem. By using a diffusion model to denoise an entire sequence of states simultaneously, the model ensures global consistency across the entire horizon, avoiding the step-by-step degradation typical of autoregressive world models.
Diffused Value Functions
Similarly, Mazoure et al. leverage this concept for reinforcement learning in Value function estimation using conditional diffusion models for control. Their framework, Diffused Value Function (DVF), uses conditional diffusion models to generate future state distributions directly from initial state sequences. By avoiding sequential autoregressive generation, DVF circumvents the accumulation of hallucinations that typically limits long-horizon model-based planning.
Strategy 3: Embedding Continuous Time and Physical Invariants
A third approach addresses the mathematical formulation of the simulator itself. Discrete-time step functions introduce arbitrary boundaries where truncation errors can accumulate. Moving to continuous-time formulations or enforcing physical conservation laws can bound the error growth.
Continuous-Time Operators
In CFO: Learning Continuous-Time PDE Dynamics via Flow-Matched Neural Operators, Hou et al. argue that discrete-time autoregressive models are structurally prone to compounding errors when simulating complex physical systems. They propose the Continuous Flow Operator (CFO), which uses flow matching to learn continuous velocity fields directly. Operating in continuous time eliminates the rigid step-to-step error propagation inherent to discrete rollouts.
Port-Hamiltonian Dynamics
To prevent energy and information from dissipating or exploding during rollouts, Hoang et al. introduce Information-preserving Graph Neural Simulators (IGNS) in Improving Long-Range Interactions in Graph Neural Simulators via Hamiltonian Dynamics. By embedding port-Hamiltonian systems directly into the network architecture, the model guarantees that energy conservation laws are respected, preventing the simulated trajectory from diverging into unphysical states over long horizons.
Hierarchical Implicit Emulators
In classical numerical analysis, implicit methods are widely used because they evaluate future states globally to ensure stability, unlike explicit forward steps. Ruoxi Jiang et al. adapt this insight to deep learning in Hierarchical Implicit Neural Emulators. By predicting a hierarchy of coarse-grained future states and solving them implicitly, their model achieves stability over sequences ten times longer than its training data, preventing the simulation collapse common to standard autoregressive rollouts.
Strategy 4: Hybrid Grounding and Filtering
When pure neural rollouts must operate over massive horizons (such as decades of climate data or complex engineering simulations), a final strategy is to introduce an external "ground truth" anchor to periodically correct the model's drift.
Pure Autoregressive:
s_0 ---------------------------------------------------------> Catastrophic Drift
Hybrid Grounding (e.g., HyPER):
s_0 ---> [ Neural Surrogate ] ---> Drift Detected ---> [ High-Fidelity Physics Call ] ---> Corrected State
Active Error Correction via Reinforcement Learning
In Model-Agnostic Knowledge Guided Correction for Improved Neural Surrogate Rollout, Srikishan et al. introduce HyPER (Hybrid PDE Predictor with Reinforcement Learning). Recognizing that neural surrogates will inevitably drift when run autoregressively, they train an RL agent alongside the surrogate. The agent monitors the rollout and intelligently decides when to "call for help" from an expensive, high-fidelity classical physics simulator, resetting the accumulated error before catastrophic failure occurs.
Causal Bayesian Filtering
For long-term macro-forecasting, the authors of Causal Climate Emulation with Bayesian Filtering, Hickman et al., present PICABU. To simulate 100 years of climate data without encountering numerical instability and divergent trajectories, they couple causal representation learning with a Bayesian filter. The filter continuously bounds the neural network's predictions using underlying causal climate modes, ensuring the rollout remains physically grounded over decades of simulation time.
Key Foundational Work
Planning in Dynamic Environments with Conditional Autoregressive Models (2018) — contributed 2 concept elements to this direction.
Graph Networks as Learnable Physics Engines for Inference and Control (2018) — contributed 1 concept elements to this direction.
Dynamic Modelling of Chaotic Time Series with Neural Networks (1994) — contributed 1 concept elements to this direction.