Symmetry Disagreement as a Drift Alarm for Long Neural Rollouts

Neural simulators predict the future one step at a time and feed each prediction back in as the next input. Small errors therefore compound: after a few thousand steps a fluid emulator can drift into states that no real flow would reach, and then blow up. Current fixes need extra learned machinery to notice drift. Thermalizer trains a diffusion model and a noise classifier; DiffusionRollout samples several stochastic futures. We propose a cheaper signal that comes from the physics. Most simulated systems have known symmetries, such as the fact that a periodic flow reflected or shifted is still a valid flow. A model that has only learned these symmetries from data should honor them on familiar inputs and break them on unfamiliar ones. So at each step we run the model on a few symmetric copies of the current state, map the predictions back, and measure how much they disagree. That disagreement is a free, training-free alarm that the rollout has left the data the model knows.

The same idea also gives a correction, borrowed from symmetry-reduced search in AI planning. Planners that search over "orbits" treat all symmetric versions of a state as one node and step from a single chosen representative. We do the analogous thing in continuous rollouts. We search over the finite set of symmetric copies, step from the copy on which the model is most self-consistent, or average over all of them, and transform back into the physical frame. The two-sentence version is this: symmetry violation is a sensor for off-distribution drift in non-equivariant emulators, and choosing a representative within the symmetry orbit is a cheap corrector that we can compare directly against diffusion-based stabilizers.

Core Claim

Let $f$ be a learned one-step emulator and $H$ a small finite set of exact symmetries of the true dynamics (for example the eight grid rotations and reflections of a periodic square domain). For a state $x$, each $g \in H$ gives a back-mapped prediction $g^{-1} f(g x)$. Let $\bar f(x)$ denote the mean of these $|H|$ predictions. We define the orbit disagreement as

$$ D(x) = \frac{1}{|H|}\sum_{g \in H} \frac{\lVert g^{-1} f(g x) - \bar f(x) \rVert^2}{\lVert \bar f(x) \rVert^2}. $$

For an exactly equivariant model, $D$ is zero. For a model trained with or without symmetry augmentation, $D$ is small on in-distribution states. The core claim is that $D$ rises measurably before a rollout diverges, because learned equivariance is itself a property that holds only on the training distribution. If this is true, we get three things at the cost of $|H|$ batched forward passes and no auxiliary network. First, a drift detector whose threshold can be calibrated from quantiles of $D$ on held-out ground-truth states rather than hand-tuned. Second, a corrector: orbit averaging, meaning replacing $f(x)$ by $\bar f(x)$, which projects the step onto its symmetric part. Third, a trigger that fires an expensive corrector such as Thermalizer only when needed.

This answers a tension raised in the file. Learned canonicalization can flip abruptly under tiny perturbations, which Equivariance with Learned Canonicalization Functions flags as a hazard for feedback loops. Our design sidesteps that hazard. We always map the prediction back to the physical frame, and the true dynamics are symmetric under $H$. As a result, switching which copy the model steps from changes only which model error we incur. It never introduces a jump in the physical state. Discontinuity of the representative choice is therefore harmless here, which is not the case when the canonicalized state itself is fed forward.

Relation to Prior Work

Two lines of work supply the method. Orbit space search in planning collapses symmetric states to canonical representatives so that search never distinguishes states that are functionally identical. Exploiting Problem Symmetries in State-Based Planners maps states to canonical forms for duplicate elimination. K* Search Over Orbit Space for Top-k Planning shows that plans found in orbit space map back exactly to plans in the original space. Learning Efficiency Meets Symmetry Breaking pairs graph-automorphism action pruning with a learned permutation-invariant embedding used as a state hash. We borrow two elements from this work: the "search within the orbit, map back" structure, and the idea that a learned representation's behavior under symmetry is informative. We change the selection criterion from a lexicographic canonical form to model self-consistency. We also change the goal from pruning to error control. None of these planning papers touches continuous rollouts or numerical drift.

Three lines of work motivate the proposal by showing that drift is real and that current detectors are costly. Thermalizer pulls drifting Kolmogorov and quasi-geostrophic states back onto the attractor with a diffusion model and a noise classifier, but it needs manually set thresholds and does not use any known symmetry. DiffusionRollout uses the spread of sampled futures as an error proxy, which requires a stochastic model and multiple samples. Mitigating Compounding Error via Video Representation Regularization shows that effective rank of hidden features collapses before visual degradation, giving an internal early-warning signal. Our proposal differs from all three because its detector is external to the model, deterministic, and grounded in exact physical structure rather than learned statistics.

Other work constrains the design. Canonical Autoregressive Generation keeps language models on the canonical tokenization seen in training at essentially no latency cost. It is the closest conceptual precedent, "restrict rollouts to canonical representatives," but its canonicity is a deterministic property of a discrete tokenizer, whereas ours is a choice among physically equivalent continuous states. Neural Field Discovery Disentangles Equivariance in Interacting Dynamical Systems warns that external fields break nominal symmetries. For that reason we restrict $H$ to symmetries that are exact for the benchmark's forcing and boundary conditions. Treating a broken symmetry as exact would make $D$ measure model bias rather than drift.

Research Program

We will build the method in three stages.

The first stage is diagnostic. We train standard non-equivariant emulators (a U-Net and a dilated ResNet, the architectures that diverge in the Thermalizer study) on Kolmogorov flow and on Gray-Scott reaction-diffusion. Each is trained in two versions, with and without symmetry augmentation. For Kolmogorov flow, $H$ is the exact discrete group of the forced flow: rotation by π, together with shift-reflections in the forcing direction and a finite set of shifts along the homogeneous direction. For Gray-Scott on a periodic square grid, $H$ is the eight-element group of grid rotations and reflections. We log $D$ along thousands of rollouts alongside ground-truth error, energy spectra, and effective rank of hidden features. The question is whether $D$ predicts divergence earlier and more reliably than competing signals.

The second stage builds correctors from the signal. Orbit averaging applies the $\bar f$ step at every step, or only when $D$ exceeds a calibrated quantile. Orbit selection steps from the copy $g^\ast = \arg\min_g$ of that copy's deviation from the orbit mean, then maps back. A hybrid uses $D$ as the trigger for Thermalizer's denoiser, replacing its learned noise classifier and its hand-set start threshold.

The third stage is a boundary test. We run the same pipeline on an exactly equivariant backbone, where $D \equiv 0$. This establishes how much of any gain is simply "being equivariant" rather than "detecting drift."

Baselines

The naive reference is the plain autoregressive U-Net rollout with no correction. Thermalizer reports that such emulators on Kolmogorov flow diverge into non-physical states within a few thousand steps, shown diverging by step 5000. Any method must at minimum push the divergence step well past that under the same training data.

The strongest existing stabilizer is Thermalizer itself (Thermalizer). It reports stable horizons extended by up to two orders of magnitude over unthermalized emulators, with its noise classifier adding roughly 60% to the cost of a single step. We will run it with the authors' published setup on the same emulator weights and the same trajectories. We will report stable horizon against wall-clock cost per simulated step. Our method "beats" it if it matches its stable horizon at lower cost per step. Alternatively, the hybrid counts as a win if replacing Thermalizer's classifier with $D$ keeps the horizon while removing the classifier network and its threshold tuning.

For detection specifically, the strongest competitor is DiffusionRollout's sample standard deviation (DiffusionRollout). That paper reports a 15.37% error reduction on Gray-Scott over a fixed 6-step advancement. No reported result was found on Lacuna for its detector's accuracy at predicting divergence. We will therefore compute it ourselves, using a diffusion emulator given the same number of samples as our $|H|$ forward passes. We also include effective rank from Mitigating Compounding Error via Video Representation Regularization as an internal early-warning baseline. No reported result was found on Lacuna for effective rank on PDE emulators.

The key ablation isolates the symmetry ingredient. It replaces the $|H|$ symmetric copies with $|H|$ copies perturbed by small random noise, matched in norm to the average distance between symmetric copies. If random-perturbation disagreement predicts divergence as well as orbit disagreement, the symmetry structure is not doing the work and the claim fails.

Two further baselines complete the set. One is an exactly equivariant backbone (a group-convolutional network over the same finite group) rolled out without correction. The other is the learned-canonicalization pipeline of Equivariance with Learned Canonicalization Functions, in which the canonicalized state is fed forward. That pipeline tests whether feeding forward a canonicalized state, rather than mapping back, suffers the flipping instability.

All methods share the same training trajectories, a matched hyperparameter search budget, and a matched inference budget reported in forward-pass equivalents.

Why not just…?

Why not just build the symmetry into the architecture? This is the most serious threat. For an exactly equivariant model, orbit averaging does nothing and $D$ carries no signal. If an equivariant network rolls out as stably as our corrected non-equivariant one, then our corrector's value reduces to cases where equivariance is not built in. The equivariant-backbone baseline settles this. There are reasons to expect the case still matters. Equivariance with Learned Canonicalization Functions reports that hard-coded group networks grow costlier with group size, while canonicalization reached 1.99% versus 2.28% error on Rotated-MNIST at near-flat cost. Many deployed large emulators are also not equivariant. The detector claim, however, stands only for such models, and we will say so.

Why not just train with symmetry augmentation and stop there? Augmentation makes the model approximately equivariant on training states, which is exactly the regime where $D$ is small. Our hypothesis is that this approximate equivariance decays off-distribution. If augmentation instead yields equivariance that holds deep into drifted states, $D$ stays flat before divergence and the detector fails. The augmented-model condition in the diagnostic stage tests this directly. It is a genuine falsification risk, not a strawman.

Why not just use Thermalizer or ensemble spread? Both work. The question is cost and calibration. Thermalizer needs a trained diffusion model and a classifier adding about 60% per step, plus hand-set thresholds. DiffusionRollout needs a stochastic emulator and multiple samples. Our detector reuses the deterministic emulator and is calibrated from quantiles on ground-truth data. If orbit disagreement is a strictly worse predictor than ensemble spread at equal forward passes, we would recommend ensembles, and the matched-budget comparison is designed to reveal exactly that.

Proposed Experiments

The decisive first experiment treats drift detection as prediction. For each rollout step on Kolmogorov flow and Gray-Scott, the label is whether the rollout leaves the physical range (Thermalizer's bounded-MSE criterion) within the next $K$ steps. We measure the area under the ROC curve, and the median lead time before divergence, for orbit disagreement, random-perturbation disagreement, diffusion sample spread, and effective rank. Orbit disagreement must beat random perturbation clearly for the proposal to proceed.

The second experiment compares stable horizons. We measure duration with correlation above 0.9 (DiffusionRollout's criterion) and time to divergence (Thermalizer's criterion) for the naive rollout, orbit averaging, orbit selection, the $D$-triggered Thermalizer hybrid, full Thermalizer, and the equivariant backbone. Results are plotted against forward-pass cost, and energy-spectrum error is checked on the long-run statistics.

The third experiment runs quasi-geostrophic turbulence as a transfer test with the smaller symmetry group appropriate to its setup. We add a deliberately broken-symmetry control in which weak non-symmetric forcing is introduced. There, $D$ should rise even on ground-truth states, confirming the warning from Neural Field Discovery Disentangles Equivariance in Interacting Dynamical Systems that the method requires symmetries to be exact.

The outcome that would kill the idea is flat orbit disagreement before divergence in augmented models, or no advantage over random-perturbation disagreement at matched cost. The outcome that would establish it is a $D$-triggered corrector that matches Thermalizer's horizon without a learned classifier and at lower per-step cost.