Parity-Safe Normalization for Neural Surrogates of Intrinsic Plasma Rotation

Tokamak plasmas spin up on their own. Nothing pushes them, but turbulence carries momentum outward in a lopsided way. In the simplest equations for that turbulence, a mirror symmetry along the magnetic field lines forces this lopsided flux, called residual stress, to cancel exactly. A nonzero value appears only when small corrections break the mirror symmetry, and those corrections are of order $\rho_*$, the ion gyration radius divided by the machine radius, roughly a few parts in a thousand. Neural surrogates now emulate full five-dimensional gyrokinetic turbulence about a thousand times faster than the numerical code they imitate. Their grids are so large that training fits only one or two samples per GPU, which is exactly the regime where batch-based normalization layers become noisy and unstable. Our central worry is that any normalization whose statistics are shared across samples (batch statistics, cross-iteration memory, running averages) quietly breaks the field-line mirror symmetry. That broken symmetry would show up as fake residual stress, possibly as large as the real $O(\rho_*)$ signal.

We propose to measure this effect directly, and to build a memory-based normalization that keeps the stability benefit of pooled statistics without the leak. The method splits each feature channel by how it behaves under the mirror transformation. Statistics for mirror-symmetric channels are pooled over a training memory and recentered as usual. Mirror-odd channels are only rescaled, using second moments symmetrized over the mirror pair, so they cannot acquire a spurious offset. The test is simple. A surrogate trained on physics that is exactly mirror-symmetric must predict zero time-averaged residual stress. When the symmetry-breaking terms are switched on, it must recover the linear dependence of residual stress on $\rho_*$ that the reference simulations show.

Background and the gap

Three bodies of work meet here, and each plays a different role.

The physics supplies both the target and the constraint. Sung et al. show that the lowest-order local gyrokinetic equations have an exact parallel parity symmetry. Retaining the poloidal derivative of the ballooning envelope, an $O(\rho_*)$ correction implemented in the GKW code, breaks it and yields a momentum flux exactly linear in $\rho_*$. For Cyclone base-case parameters this predicts a rotation gradient near 0.38. Hornsby et al. show that global effects, chiefly profile shearing and density curvature, matter even more for ASDEX Upgrade: global GKW gives $u'\sim -0.55$ against $-0.043$ for local flux tubes. Camenen et al. show that up-down asymmetric geometry is a third symmetry-breaking route. All three establish that the quantity of interest is a small, symmetry-protected residue. We use them to define the symmetry operation and the reference behaviour, not as surrogate methods.

The surrogate side motivates the work. GyroSwin and its precursor 5D Neural Surrogates emulate the 5D distribution function, potential and heat flux on grids such as $32\times8\times16\times256\times32$. They report roughly 200 to 1,000 times speedups and stable rollouts beyond 100 steps. Both work under adiabatic electrons and target heat flux and zonal flows. Neither predicts momentum flux, and neither examines how its normalization layers interact with symmetry. We build on their architecture and differ in target (residual stress) and in the property audited (parity equivariance).

The normalization literature supplies the mechanism and its failure modes. Cross-iteration BN (Yao et al.) pools statistics over the last $k$ steps with a Taylor correction for weight drift. Momentum BN (Yong et al.) uses scheduled moving averages and argues that batch noise is a regularizer whose strength scales inversely with batch size. Kalman Normalization (Wang et al.) shares statistics across depth. Two further papers diagnose the costs of shared statistics. Instance Enhancement BN names "feature pollution," where one sample is normalized by unrelated others. Huang et al. show that the mismatch between training and inference statistics compounds layer by layer. Batch-free alternatives such as Filter Response Normalization avoid batch coupling but pool uniformly over space.

None of this work asks whether pooled statistics respect a discrete symmetry of the target, because image labels have no sign-flip structure. We take feature pollution seriously in a setting where it has a precise, measurable signature: an antisymmetric bias in a symmetry-protected output.

Core Claim

Let $P$ be the field-line mirror operation of the lowest-order equations: reflect the parallel coordinate and the parallel velocity together, with the accompanying sign flips of parity-odd fields. Let $M$ be the surrogate's one-step map. Define the parity defect on an input $x$ as

$$D(x) = \frac{\lVert M(Px) - P\,M(x) \rVert}{\lVert M(x) \rVert}.$$

For a model trained on data generated without symmetry-breaking terms, the ideal $D$ is zero. Any time-averaged residual stress the model predicts is then pure artefact.

Our claim has two parts. First, normalizations whose mean is shared across samples have an irreducible nonzero $D$. Standard BN at small batch, CBN, MBN and SyncBN all fall in this class. A shared mean subtracted from a parity-odd channel does not flip sign when the input is mirrored. We expect the resulting fake residual stress to be comparable to the $O(\rho_*)$ physical signal at $\rho_*$ values typical of present devices. Second, a parity-split memory normalization keeps $D$ at numerical precision while retaining the optimization stability that pooled statistics give at $B=1$. It recenters only parity-even channels from a memory of past steps, and rescales parity-odd channels by a mirror-symmetrized second moment. On held-out data, this should yield a better-resolved $\rho_*$ scaling of residual stress than either shared-statistics BN or batch-free per-sample normalization.

The second part is the one that could fail. If per-sample RMS normalization does just as well, the useful conclusion is "never share statistics in symmetry-protected surrogates," and the memory component is unnecessary.

Research Program

Data. We first generate a dedicated GKW dataset. Training on the existing 6 TB GyroSwin corpus would not suffice, because it lacks momentum-flux diagnostics and was not produced with the symmetry-breaking terms toggled. The new runs are local flux-tube Cyclone base case simulations with adiabatic electrons, matching the surrogate literature, at full 5D resolution so that the $B=1$–2 memory constraint is real. One arm runs with the $O(\rho_*)$ parallel-derivative terms of Sung et al. switched off, which makes the physics exactly mirror-symmetric. The other arm switches them on, scanning $\rho_*$ over roughly a factor of four, plus a few variations in safety factor and magnetic shear, where Sung et al. report increasing transport. A small up-down asymmetric geometry set, following Camenen et al., gives a second, geometrically different symmetry-breaking route for out-of-distribution testing. Global profile-shearing effects as in Hornsby et al. are out of scope for the first study, since local surrogates cannot represent them. This is a stated limitation, not a hidden one.

Surrogate. We use a GyroSwin-style 5D shifted-window transformer. The only variable across arms is the normalization layer, and every arm has the same compute, data, schedule search budget and rollout length.

Method. Channels are tagged as parity-even or parity-odd at the input embedding. This tagging is propagated through the network by constraining the linear layers to not mix the two classes, except through products of odd channels, which yield even channels. Even channels are normalized with memory statistics, using either the momentum schedule of Yong et al. or a CBN-style window. Odd channels are divided by a memory estimate of $\tfrac12(\mathbb{E}[h^2]+\mathbb{E}[(Ph)^2])$ and never recentered.

Analysis. We measure $D$ layer by layer, adapting Huang et al.'s estimation-shift analysis to ask whether parity defects compound with depth the way estimation shift does. We also record the time-averaged residual stress from autoregressive rollouts, computed from predicted $f$ and $\phi$ exactly as GKW computes it.

Baselines

The naive reference is the GyroSwin-style surrogate with whatever normalization a practitioner would use by default: LayerNorm per token, trained at $B=1$ with no symmetry handling. It sets the floor for both rollout accuracy and parity defect. It is run with the identical data, schedule and tuning budget as every other arm.

The strongest shared-statistics competitor is cross-iteration BN (Yao et al.), the best memory method on the file's benchmarks. On ImageNet with ResNet-50 at batch size 2 it reports 69.6% top-1, against 69.1% for Group Normalization and 55.9% for standard BN. Momentum BN (Yong et al.) reports a 2.5-point gain at $B=4$ and is run alongside it. To confirm our implementation is faithful, we first reproduce CBN's ResNet-50 batch-size-2 result to within a point before porting it to 5D. For momentum flux in a neural surrogate there is no reported result found on Lacuna, so all gyrokinetic comparisons are against numbers we generate ourselves.

The strongest batch-free competitor is per-sample RMS normalization, together with Filter Response Normalization (Singh & Krishnan). RMS rescaling without centering is already sign-invariant, so it is the most dangerous rival on the parity question.

The key ablation removes memory from our method, keeping parity splitting but using per-sample statistics. A second ablation keeps memory but drops the split. Together they isolate whether the gain, if any, comes from the symmetry handling or from the pooling.

Beating the field means meeting two conditions on held-out runs. First, on the symmetric arm, our method must cut the time-averaged residual-stress magnitude below the statistical noise floor of the GKW reference ensemble, while CBN and MBN do not. Second, on the symmetry-breaking arm, the fitted slope of residual stress against $\rho_*$ must lie closer to the GKW slope than every baseline's, with heat-flux error no worse than the LayerNorm floor.

Why not just…?

Why not just use per-sample LayerNorm or RMSNorm, which never share statistics? This is the real threat. RMS rescaling is sign-invariant, so it may already be close to parity-safe, and transformers default to it anyway. The file's evidence that batch-free layers lose the regularizing effect of pooled noise comes entirely from image classification. Examples are the 69.1% versus 69.6% gap in Yao et al. and the regularization argument of Yong et al., and neither may transfer to autoregressive physics emulation. The memory-free ablation settles this directly. If it matches the full method on both the symmetric-arm defect and the $\rho_*$ slope, we report that, and the recommendation becomes "use per-sample, uncentered normalization."

Why not just symmetrize the output or augment with mirrored samples? Averaging the residual-stress prediction over the mirror pair, $\tfrac12(\Pi(x)-\Pi(Px))$, forces exact antisymmetry at twice the inference cost. Mirror augmentation encourages equivariance more cheaply. Both fix the output after the fact. Neither stops shared statistics from injecting sample-to-sample noise into internal features, which would then raise the variance of the small antisymmetric signal rather than bias it. We include output symmetrization stacked on top of each normalization arm. If CBN plus symmetrization matches our method on $\rho_*$-slope error and variance across seeds, the normalization change is not needed and we will say so.

Why not just use SyncBN or gradient accumulation to get a large effective batch? Gradient accumulation changes only the optimizer step. Each forward pass is still normalized with one- or two-sample statistics, so feature pollution and the compounding estimation shift described by Huang et al. remain. SyncBN pools across devices, which is costly at billion-parameter, hundreds-of-megabyte-per-sample scale. More to the point, it is still a shared mean, so our claim predicts it leaks parity just like small-batch BN. A two-node SyncBN arm at an effective batch of 16 tests that prediction directly.

Proposed Experiments

The decisive experiment is the symmetric-arm null test. Every normalization variant is trained on mirror-symmetric GKW data and rolled out for at least 100 steps on unseen seeds. We then compare the time-averaged residual stress to the spread of the GKW ensemble, and report the per-layer parity defect $D$. A clear separation between shared-mean methods and parity-split or per-sample methods would confirm the first half of the claim. No separation would refute it.

The second experiment is the $\rho_*$ scan. We fit the slope of residual stress against $\rho_*$ for each arm and compare it with the GKW slope and with the linear law of Sung et al.

The third tests transfer to the up-down asymmetric geometry runs. Here symmetry breaking comes from a different source, and an artefact-free surrogate should change its residual stress sign and magnitude with the geometry, not with the normalization choice.

Throughout, heat-flux error and rollout stability are reported as guardrails, so that a parity-safe method cannot win by simply predicting less. Wall-clock overhead is reported against CBN's stated cost of about 11% extra compute and 7% extra memory.