Treating Rollout Horizons as Competing Tasks

A neural simulator that predicts the next state of a physical system can be accurate one step ahead and still fail badly when its predictions are fed back in as inputs for hundreds of steps. The standard fix is unrolled training: run the model on its own outputs for several steps during training and sum the prediction losses at each step. This note argues that the sum hides a conflict. The loss at step 1 and the loss at step 8 are different objectives that happen to share one set of weights. The step-1 loss rewards a sharp one-step fit. The step-8 loss, whose gradient has passed back through eight copies of the model, rewards dynamics that do not amplify small errors. Multi-task learning, the study of training one network on several objectives at once, has shown that such objectives can differ widely in gradient size and can point in opposing directions, and that naive summing then lets one objective erase another.

We propose to measure this conflict directly and to resolve it during training. For each rollout step $k$ we compute its gradient separately. We rescale the gradients so that no single step dominates, and we remove the part of any near-horizon gradient that points against the long-horizon gradient. The claim is specific and can be falsified. Conflict between horizons should be rare for stable, dissipative systems and frequent for chaotic ones. The direction correction should therefore help mainly on chaotic systems. On those systems it should extend stable rollout length beyond what unrolled training achieves with equal compute, and beyond what rescaling gradient sizes alone achieves.

Core Claim and Positioning

The closest prior work plays three distinct roles. The first group motivates the proposal. Message Passing Neural PDE Solvers introduced the pushforward trick: unroll the model on its own outputs but backpropagate only the loss at a later step. APEBench showed across many procedurally generated PDEs that training on unrolled windows prevents blow-up over 100+ steps. Learning with Imperfect Models analyzes, for linear systems, a single-step model trained on a joint loss summed over several horizons. All three combine horizon losses as a fixed scalar sum and never examine how the per-horizon gradients relate to each other. That relationship is our object of study. Investigating Compounding Prediction Errors in Learned Dynamics Models supplies the physical reason to expect conflict. It shows that whether error plateaus or grows exponentially depends on the system's stability, and that for unstable systems adding data from 10 to 100 trajectories gives diminishing returns. Gradients that pass through many unstable Jacobians should behave very differently from one-step gradients, and more data does not remove that difference.

The second role is method. RotoGrad homogenizes task gradients in two ways. It reweights gradient magnitudes to favor slower-learning tasks, and it rotates the shared feature space for each task to reduce directional cancellation. We reuse the magnitude part almost as-is, treating each horizon as a task. We do not reuse per-task feature rotations, because a horizon-specific rotation would make the one-step dynamics model depend on the step index. That would break the model's use as a single recursive map beyond the training window. We replace rotation with an asymmetric projection applied directly to parameter gradients.

The third role is constraint and contrast. HERO combines ground-truth supervision with relative supervision against failure modes of a lagged checkpoint. It explicitly avoided multi-objective gradient conflict by folding everything into a sample-dependent reweighting of one gradient. We read this as evidence that its authors ran into conflict and routed around it. We confront the conflict instead, and because the two mechanisms act on different quantities, we also test whether they stack.

The main method differs from each neighbor in a clear way. It keeps the recursive single-step model, unlike direct multi-step predictors (Combating the Compounding-Error Problem with a Multi-step Model) or continuous-time flows (CFO). It adds no architecture, unlike Hamiltonian inductive biases (Improving Long-Range Interactions in Graph Neural Simulators via Hamiltonian Dynamics). It changes only how the gradients from different horizons are combined.

Method

The model $f_\theta$ maps a state to the next state. It is unrolled $K$ steps from a true initial state, giving losses $L_1, \dots, L_K$ against ground truth. Standard unrolled training steps along $\sum_k \nabla_\theta L_k$. We compute each $g_k = \nabla_\theta L_k$ separately; for the $K \le 8$ windows used on 1D and 2D grids this costs roughly $K$ backward passes. Each $g_k$ is first rescaled to a common norm, weighted toward horizons whose loss is falling slowest, as in RotoGrad's magnitude homogenization. Then, for every near horizon $j$ whose gradient conflicts with the aggregate long-horizon gradient $G = \sum_{k > K/2} g_k$, we remove the opposing component:

$$ g_j \leftarrow g_j - \frac{\min(0,\; g_j^\top G)}{\|G\|^2}\, G , $$

Here "conflicts" means $g_j^\top G < 0$. The update is the sum of the corrected gradients.

The projection is asymmetric on purpose. In this setting a one-step error is cheap, because the next step can correct it. Instability is expensive, because nothing corrects it. Near-horizon gradients may therefore still move the model in any direction that does not undo long-horizon stability. Long-horizon gradients are never altered. We will also run a symmetric variant, to test whether the asymmetry matters or whether any conflict removal gives the same effect.

Research Program

The first stage is diagnostic, and the project depends on it. We train standard unrolled models on a stability-ordered subset of APEBench dynamics: diffusion and advection (stable), dispersion and Burgers (shock-forming), and Kuramoto–Sivashinsky and Kolmogorov flow (chaotic). Throughout training we log the pairwise cosine similarity and norm ratio of the per-horizon gradients.

The hypothesis predicts two things. The fraction of steps with negative cosine between near and far horizons should rise with the system's instability. The norm ratio should drift in opposite directions for dissipative and chaotic dynamics: far-horizon gradients should vanish on dissipative systems and explode on chaotic ones. If conflict turns out to be rare everywhere, the direction correction has nothing to fix. The paper then reduces to a characterization of horizon-gradient magnitudes, and we would say so.

The second stage trains the homogenized models and compares them against the baselines below. The third stage relates the gains back to the diagnostics. For each dynamics, we ask whether improvement from the direction correction is predicted by that dynamics' measured conflict rate.

Baselines

The naive reference is a Fourier neural operator trained by one-step teacher forcing, meaning training only on true input states. HERO reports its 1D Burgers 100-step normalized error as 0.541 (HERO). We will reproduce this number in our pipeline before comparing anything else, so that our setup is anchored to the published one.

The unrolled-training baselines are the pushforward trick from Message Passing Neural PDE Solvers and full summed unrolling as analyzed in APEBench. The latter is the same objective as the hybrid multi-step loss in Learning with Imperfect Models. This is the most direct competitor, because our method optimizes exactly the same losses and differs only in how their gradients are combined. Its unroll depth $K$ is tuned over the same grid as ours.

The strongest existing method is HERO. It reports nRMSE@100 of 0.153 on 1D Burgers, and a stable rollout length within a 10% error margin of 26.6 steps versus 8.2 for the baseline on 2D anisotropic diffusion (HERO). We will run HERO in our codebase on identical data splits and backbone. We will report the published figures alongside our reproduction so readers can see any gap.

Because our method costs about $K$ backward passes per update, fairness is enforced in GPU-hours rather than in number of updates. Every baseline receives the same wall-clock budget, which it can spend on longer training or deeper unrolls. Every method receives the same hyperparameter search budget of trials.

Three ablations isolate the key ingredient. Magnitude-only homogenization tests whether rescaling alone explains the gains. Symmetric projection tests the asymmetry. A control that removes a random component of matched norm from the near-horizon gradients tests whether the gain comes from conflict structure or merely from injecting noise.

Beating the method means two things. On chaotic dynamics, it must extend stable rollout length and lower nRMSE@100 relative to summed unrolling and to magnitude-only homogenization, across seeds and at equal GPU-hours. It must also either beat HERO or improve on HERO when the two are combined. On stable dynamics we predict no gain, and we would count a gain there as evidence against our proposed mechanism rather than for the method.

Why not just…?

Why not just use the pushforward trick or a deeper unroll? Pushforward is a special case of our setting: it keeps only one horizon's gradient and discards the rest. Deeper unrolls add more horizons to a sum whose conflicts we hypothesize grow with depth. HERO reports that pushforward alone left spectral distortion and energy drift unresolved on Burgers (HERO). The equal-GPU-hour comparison, with $K$ tuned for the baselines, will show directly whether these baselines can match our method when given our budget.

Why not just reweight the horizon losses? This is the most serious threat. If the only problem is that far-horizon gradients explode or vanish, then norm equalization, or even a hand-tuned per-horizon weight schedule, would capture the full benefit. Our design treats this as an explicit hypothesis rather than a strawman. The magnitude-only ablation and a tuned fixed-weight schedule are both run. If either matches the full method on chaotic dynamics, we conclude that the problem is magnitude rather than direction and report exactly that.

Why not just use direct multi-step predictors? Combating the Compounding-Error Problem with a Multi-step Model removes feedback by learning one predictor per horizon, each conditioned on the true initial state. This works in low-dimensional control but grows with horizon, and it cannot roll past its trained horizon. We keep a single recursive map that can be run indefinitely. A shared-backbone multi-head variant will be included at small horizon on the 1D tasks, as a reference for how much compounding a direct model avoids.

Why not just regularize the hidden representations, as video models do? Mitigating Compounding Error via Video Representation Regularization shows that rollout collapse in video diffusion follows a drop in the effective rank of hidden states, and it counters this with an isotropic prior. That acts on features, whereas we act on how the objectives are combined. The two are not exclusive. We will log effective rank under every method to see whether direction correction preserves rank without any explicit prior, and we will also run the isotropic regularizer as a combinable add-on.

Decisive Experiments

The first decisive check is the stage-one diagnostic: near–far gradient conflict should rise from diffusion through Burgers to Kuramoto–Sivashinsky and Kolmogorov flow. The second is the stage-two comparison on those chaotic systems: at equal GPU-hours, full homogenization should beat both summed unrolling and magnitude-only homogenization on stable rollout length and nRMSE@100, and the random-projection control should not reproduce the gain. The third is the stage-three regression across the APEBench subset: the size of each dynamics' improvement should be predicted by its measured conflict rate.

Supporting metrics are HERO's spectral discrepancy and energy drift diagnostics, together with Burgers nRMSE@100 compared against HERO's 0.153. If the conflict-rate pattern appears but the direction correction gives nothing over rescaling alone, the result still stands as a characterization of how horizon gradients behave across stability regimes. If the conflict-rate pattern does not appear, the hypothesis is wrong.