Trajectory-Calibrated Deferral for Autoregressive Forecasters with Directional Error

Autoregressive forecasters (models that predict the next state and feed that prediction back in as input) drift. Small errors push the model onto inputs it never saw in training, and the errors grow. Two further facts make this worse. First, the model's own uncertainty becomes more overconfident the longer it rolls forward, so the signal you would use to stop it fails exactly when you need it. Second, the damage is usually one-sided: a turbulence emulator loses energy, a concentration field goes negative, a pollution forecast misses peaks instead of overshooting them. We propose a gate that decides, at each step, whether to keep trusting the emulator or to defer, meaning resynchronize the state from an expensive trusted source such as a numerical solver or a fresh observation. The gate is trained on the emulator's own rollout errors rather than its reported variance. It scores the one-sided failure the user actually cares about. Its threshold is calibrated on whole trajectories, which yields a finite-sample guarantee on how often an entire rollout violates a directional error cap, for a stated budget of deferrals.

In two sentences: stop asking a drifting model how confident it is, and instead train a separate critic to predict how badly its next few steps will fail in the harmful direction. Then pick the critic's alarm threshold on held-out full trajectories, so that "fraction of rollouts that ever cross the error cap" is certified below a user target while using as few expensive corrections as possible.

Core Claim

Selective prediction, the practice of letting a model abstain and hand hard cases to a fallback, has good guarantees for independent queries. Selective Classification for Deep Neural Networks inverts a binomial tail bound on a validation split to certify 0/1 risk. Conformalized Selective Regression abstains when a conformal quantile interval is too wide. Neither has a notion of state. In a rollout, abstaining at step $t$ changes every later input. The rollout controllers that do handle state give no such guarantees. DiffusionRollout shrinks its step size when the standard deviation across diffusion samples exceeds a hand-set tolerance. Thermalizer triggers denoising back onto the attractor when a noise classifier fires, but it assumes Gaussian errors and has brittle manual thresholds. ARROW learns a reinforcement-learning step scheduler against RMSE. All three measure drift with symmetric quantities. None certifies anything about the corrected trajectory.

Our claim has three parts, each of which can be tested on its own:

  1. The gate should be a critic trained on the model's own rollouts. Predictive variance degrades with depth. Beyond Accuracy: Are Time Series Foundation Models Well-Calibrated? documents growing overconfidence with autoregressive depth and shows that Gaussian heads hurt calibration on skewed data.
  2. The critic should score a user-specified directional liability rather than a symmetric width. Rollout residuals become increasingly skewed as depth grows. This is a hypothesis we measure directly; it is motivated by the energy loss seen in Thermalizer and the smoothing seen in ProbTS.
  3. The threshold should be calibrated with whole trajectories as the unit. The gate's own resets are then already baked into each calibration sample, so the feedback problem disappears from the guarantee.

Research Program

Let $s_t$ be the true state, $\hat s_t$ the emulator state under the gated policy, and $L(\hat s_t, s_t) \ge 0$ a liability chosen per domain. Examples are the kinetic-energy deficit plus field error for Kolmogorov flow, negative-mass magnitude for Gray-Scott, and underprediction of PM2.5 above a health threshold.

The critic $c_\theta$ reads the current emulator state together with cheap rollout features: steps since the last reset, spectral summaries, and the emulator's own head outputs. It outputs an upper quantile of the maximum liability over the next $h$ steps. We train it with a pinball loss on liabilities collected by running the base emulator freely from many initial conditions. It is optionally iterated once on rollouts generated under a preliminary gate, so that it sees the reset-induced state distribution.

The policy defers whenever $c_\theta(\hat s_t) > \tau$, and each deferral costs one call to the trusted source. For a threshold $\tau$, a trajectory of horizon $T$ is a violation if

$$V(\tau) = \mathbb{1}\left[\max_{t \le T} L(\hat s_t, s_t) > r^*\right],$$

where $r^*$ is the user's error cap. Because $V(\tau)$ is a 0/1 event per independent trajectory, the binomial tail inversion of SGR from Selective Classification for Deep Neural Networks applies essentially unchanged. We search thresholds from least to most conservative with a union-bound correction over the search steps. We return the largest $\tau$, which uses the fewest deferrals, such that

$$P\left(\mathbb{E}[V(\tau)] \le \alpha\right) \ge 1 - \delta,$$

where $\alpha$ is the tolerated fraction of failed trajectories and $\delta$ the failure probability of the certificate.

The guarantee is only as good as exchangeability of initial conditions between calibration and test. We state that plainly, and we test it under deliberate shift. The contribution relative to the file's Gap 3 is modest but real: it gives finite-sample, distribution-free, sequential risk control for continuous, skewed errors. It does this without proving new surrogate-consistency results, by moving the guarantee onto a trajectory-level binary event.

The distributional-symmetry direction enters as a measurable hypothesis and as an ablation, not as a new distribution family. We will measure the skewness of rollout residuals projected onto the liability direction as a function of depth. The question is whether symmetric scores like cross-sample standard deviation waste deferral budget on harmless spread. Relaxed Quantile Regression shows that symmetric tail allocation inflates intervals under skewed noise. Unilateral boundary time series forecasting shows one-sided errors should not be penalized symmetrically. We carry both lessons into the gate's score.

Baselines

Naive references. The "do nothing" reference is free rollout of the base emulator with no deferral. Thermalizer reports that such emulators on Kolmogorov flow diverge into non-physical states by step 5000. The naive deferral reference is periodic resynchronization every $k$ steps, with $k$ set so the number of solver calls equals our method's budget. This is the baseline we fear most in chaotic systems, because error growth there is largely a function of elapsed time. If a clock beats a state-dependent critic, the critic is not worth building.

Strongest existing methods. The strongest existing uncertainty-gated rollout is DiffusionRollout's cross-sample standard deviation. It reports a 15.37% error reduction over fixed 6-step advancement on Gray-Scott. We reuse that standard deviation as a deferral score and calibrate it with the same trajectory-level procedure, which isolates the score from the calibration. The strongest drift detector is Thermalizer's noise classifier, used in the same way as a deferral score. Thermalizer reports up to roughly 100x longer stable horizons, with the classifier adding about 60% to single-step cost. For static selective regression we apply CSR's symmetric conformal interval width per step. CSR reports an error-coverage AUC of 0.328 on Communities, a static dataset; no reported result was found on Lacuna for any of these gates under a deferral budget on rollouts.

Fair comparison. Every method uses the same base emulator and the same calibration trajectories. Each score gets the same critic-training compute where one is trained. Each receives the same threshold search, and each is compared at matched deferral counts.

Ablations. Three ablations isolate the key ingredients. The first swaps the directional liability for symmetric $L_2$ inside our own critic. The second replaces the rollout-trained critic with the emulator's own predictive quantiles. The third replaces trajectory-level calibration with per-step split conformal.

What counts as beating a baseline. Beating a baseline means a strictly smaller deferral budget at the same certified $\alpha$, with empirical violation rate at or below $\alpha$ on test trajectories.

Why not just…?

Why not just reset on a fixed schedule? It is the real threat, and it is also the most honest baseline. For stationary chaotic flows the optimal policy may be nearly periodic. We therefore report the budget ratio of our critic to the best periodic $k$ at equal certified risk. The critic's advantage should appear where hazard varies with state: phase-separation fronts in Cahn-Hilliard, and pollution episodes in Beijing PM2.5. If the ratio is near one everywhere, we will report that state-dependent deferral is unnecessary for these systems.

Why not just calibrate the emulator's own variance or sample spread per horizon step? Per-horizon conformal calibration fixes average miscalibration. It cannot fix a score that ranks states poorly. The Autoregressive Penalty in Beyond Accuracy suggests exactly this: overconfidence grows on the trajectories that are drifting most. Per-step guarantees also ignore that deferring changes future states. The second and third ablations settle this directly.

Why not just use an asymmetric interval, calibrating each tail separately, instead of a directional liability? This alternative handles skew cheaply and may capture most of the gain from asymmetry. We include it as the asymmetric-score variant of the CSR baseline. If it matches our critic, the directional-liability part of the claim is redundant and the contribution reduces to rollout-trained scoring plus trajectory calibration.

Why not just avoid autoregression with direct multi-horizon prediction? ProbTS finds that non-autoregressive models smooth local dynamics. WeatherBench reports a direct CNN at about 626 m²/s² RMSE on 3-day Z500, against about 154 for operational IFS. Many emulator uses, such as long climate-statistics runs, also require stepping.

Proposed Experiments

We use four testbeds. Kolmogorov flow and two-layer quasi-geostrophic turbulence follow the Thermalizer setup; there deferral means injecting the solver state, and the liability is energy-spectrum deficit plus field error. Gray-Scott and Cahn-Hilliard follow the DiffusionRollout setup, with liabilities of negative concentration and front-position error. Beijing PM2.5 serves as a real, one-sided case. There deferral means reading the next observation, the liability is underprediction above a threshold, and the base model is the unilateral GRU, which reports MAE 13.018 against 24.444 for a standard GRU.

Metrics are three. The first is the certified and empirical trajectory violation rate. The second is the number of deferrals needed to reach $\alpha \in \{0.05, 0.1\}$. The third is the curve of deferral budget against violation rate, which replaces CSR's static ideal-point distance with a trajectory-level analogue.

Three checks are decisive:

  1. Skew diagnostic. Rollout residual skewness along the liability direction should grow with depth. If it does not, the directional component is dropped.
  2. Ranking check. At matched budget, the rollout-trained critic should beat both the emulator's variance and the ensemble spread, each calibrated identically.
  3. Coverage under shift. Empirical violation should stay at or below $\alpha$ on in-distribution test trajectories. We also report how far it degrades under shifted initial conditions, such as a changed Reynolds number or a different pollution season. That degradation tells users when the certificate can no longer be trusted.