Neural PDE solvers are networks trained to replace expensive numerical simulators, such as the Fourier Neural Operator (FNO), which learns its update rule as multiplications on Fourier coefficients. They are increasingly compressed by quantization, which stores weights and activations with few bits so the model runs cheaper. Today, a quantized solver is judged by its average squared error against the reference simulation. A physicist cares about something else: whether the compressed model still conserves mass, dissipates energy at the right rate, and keeps the correct distribution of energy across length scales. These can fail while average error barely moves, because a small, systematic bias repeated over hundreds of rollout steps compounds into a visible drift. This note proposes to measure that failure directly and to use it when deciding which weights get more bits.
The idea fits in two sentences. In an FNO, each block of spectral weights is tied to a specific wavenumber, so the parameters already have a physical reading. We score each block by how much quantizing it damages a chosen set of physical diagnostics over a multi-step rollout, keep the most damaging few percent in high precision, and push the rest to 2–3 bits. We also test whether the "performance cliff" seen below 3 bits in language models has a counterpart in neural operators that shows up first as broken physics rather than as raised error.
Core Claim
We claim three things. Below 4 bits, quantized FNOs violate conserved quantities and spectral statistics well before their one-step error rises enough to flag a problem. The damage concentrates in identifiable spectral blocks, such as the zero-wavenumber block that governs the field mean and the highest retained modes that govern dissipation. Allocating precision by rollout-level physical damage beats allocating it by one-step error or by second-order (curvature-based) error at the same average bit budget, on both conservation and long-horizon accuracy.
The second claim makes a crisp, falsifiable prediction. If rounding error in the zero-mode weights is the dominant source of mass drift, then mass drift should grow roughly linearly in rollout length, because a fixed per-step bias accumulates additively. It should also largely disappear when only those few weights are kept in full precision.
Relation to Prior Work
The closest work is Task-Circuit Quantization, referred to below as TACQ. TACQ scores language-model weights by the expected loss change when they are quantized, combined with gradient-magnitude saliency. It keeps the top weights in 16-bit precision and quantizes everything else within GPTQ, a standard post-training quantization method. It reports recovering 96% of unquantized MMLU accuracy at 3.1 bits on Llama-3-8B-Instruct, a 5.25-point gain over SpQR, another method that protects salient weights. We reuse its quantize-and-measure saliency idea, which makes TACQ the method supplier here. Our version differs in two ways. The quantity being protected is a set of physical invariants measured over a rollout rather than a next-token loss. The units being protected are wavenumber-indexed spectral blocks with physical meaning rather than anonymous weights.
Through a Compressed Lens motivates the diagnostic half of this project. It found that 4-bit post-training quantization can wreck factual recall in language models while leaving surface behavior intact; recall on Qwen2.5-14B fell from 73.08% to 25.20% under 4-bit GPTQ. The damage was localized to the final layers. That study is diagnostic only, works on text, and stays at 4 and 8 bits. We ask the analogous localization question for operators at 2–3 bits and then act on the answer.
In the scientific setting, Efficient Neural PDE-Solvers using Quantization Aware Training shows that quantization-aware training of FNOs, UNets and Transformers keeps fine turbulent structure. It also shows that quantized high-resolution models beat Float32 models run at lower resolution for the same compute. However, it reports only error and compute curves, never checks conservation, and stops at 4 bits. Adaptive Mesh Quantization assigns 8 or 4 bits to mesh nodes using a learned loss predictor. Its allocation is over spatial locations and is driven by loss, whereas ours is over parameter blocks and is driven by physics. The two are complementary rather than competing.
Two works constrain the design. QuIP rotates weights by random orthogonal matrices to spread outliers before 2-bit rounding. Such a rotation mixes wavenumber blocks, which would erase the physical reading our method depends on, so QuIP serves as a baseline and cannot be a component. Understanding intermediate layers using linear classifier probes warns that linear probes miss information that is encoded nonlinearly. For that reason we do not use probes. We measure physical quantities on the model's own outputs, where no decoder is needed.
The FNO itself comes from Fourier Neural Operator for Parametric Partial Differential Equations. The broader motivation, that scientific models must be physically consistent and not merely accurate, comes from Direction 1 work such as Deep learning in fluid dynamics.
Research Program
The first stage is diagnosis. We train full-precision FNOs on the benchmarks listed below and quantize them uniformly at 8, 4, 3 and 2 bits, using both post-training quantization and quantization-aware training. For each model we track one-step error alongside a set of physical diagnostics matched to each equation. For 1D Burgers' on a periodic domain these are total mass, which should be conserved, and kinetic energy, which should decrease monotonically under viscosity. For 2D Navier–Stokes in vorticity form they are mean vorticity, enstrophy over time, and the energy spectrum, with the error measured as log-spectral distance per wavenumber band. For 2D Darcy flow, which is steady and has no rollout, the diagnostic is the PDE residual of the predicted solution. For diffusion–sorption it is mass balance.
We also include a negative control. Spectral multiplication and pointwise activations commute with spatial shifts, so quantization with shared per-tensor scales should leave translation equivariance exactly intact. If equivariance error appears, that indicates an implementation bug, not a finding.
The second stage is attribution. We partition the network into blocks, one block per spectral weight matrix for each retained mode $k$ and layer $\ell$, plus the pointwise bypass weights. For each block $b$ we define a saliency
$$ S_b = \mathbb{E}_{x}\left[\sum_{j} w_j \,\big| D_j(\text{rollout}_T(x;\, Q_b)) - D_j(\text{rollout}_T(x)) \big| \right]. $$
In this expression, $Q_b$ quantizes only block $b$ to the target bit width, $D_j$ is the $j$-th physical diagnostic computed over a rollout of $T$ steps from initial condition $x$, and $w_j$ normalizes each diagnostic by its variance across the ground-truth data. Plotting $S_b$ against wavenumber and layer gives a map of which parameter blocks hold up which physical property. This map is the interpretability output of the project, and its predictions can be tested directly, for example "the zero-mode blocks carry mass, and the top modes carry dissipation."
The third stage is allocation. We keep the blocks with the highest $S_b$ in 8-bit or 16-bit precision and quantize the rest to 2 or 3 bits. We tune the protected fraction so that the average bit width, counting the protected blocks, matches each baseline exactly. To keep the cost of computing $S_b$ manageable, we compute it with short rollouts and group nearby high modes together. Its cost is reported against quantization-aware training wall-clock time.
Baselines
The naive reference point is uniform quantization at the target bit width with no protected weights. We run it as post-training quantization and also as straight-through quantization-aware training, following Efficient Neural PDE-Solvers using Quantization Aware Training, with the same training steps and learning-rate sweep given to our method. That paper reports its results only as Pareto plots: the file records no numeric error values and notes that 8-bit weights and activations gave "virtually no increase" in error. We therefore re-run it ourselves. No reported result was found on Lacuna for any quantized FNO below 4 bits.
The strongest existing method is TACQ-style saliency transplanted to operators. It uses the same quantize-and-measure procedure and the same protected fraction, but its objective is one-step error, as TACQ uses task loss. TACQ's language-model results above show that this family beats Hessian-based protection. Here it receives the same calibration set and the same compute budget as our method. This comparison doubles as the key ablation, because the only difference is the objective inside $S_b$, which isolates whether rollout-level physical diagnostics add anything over generic error.
A second ablation replaces the diagnostics with rollout-level error over the same horizon $T$. This separates the benefit of a longer horizon from the benefit of measuring physics.
We also run a Hessian-weighted outlier-protection baseline in the spirit of SpQR and CRVQ, which protects about 2% of channels. CRVQ reports 9.81 perplexity on LLaMA2-13B at about 1.06 bits. Finally, we run QuIP-style incoherence rotation, for which QuIP reports 6.33 WikiText2 perplexity at 2 bits on Llama 2 70B. Both are language-model numbers. They establish that these methods are strong in their home domain and are not targets to reproduce on operators.
Beating a baseline means achieving lower conservation drift and lower spectral distance at the end of the rollout at the same average bit width, without worse one-step error. The comparison uses three training seeds, and the gap must exceed the seed-to-seed spread.
Why not just…?
Why not just keep the lowest Fourier modes in high precision? This heuristic is free and physically sensible, and it is the most serious threat to the proposal. If attribution only ever recovers "low modes matter," the saliency map adds nothing. We include the heuristic at a matched bit budget. Our prediction is that it protects mass and large-scale energy but misses the high-mode blocks that control dissipation, so its spectral error should grow at high wavenumbers over long rollouts. If that prediction fails on Navier–Stokes, we will report the heuristic as sufficient.
Why not just project the output back onto the conserved quantities after each step? For a linear invariant such as mass, subtracting the drift is trivial and would make the mass result uninteresting. We therefore run projection as a baseline and treat mass as a sanity check rather than the headline result. Projection cannot correct the shape of the energy spectrum or the timing of dissipation without a model of what the correct values are. The decisive comparison is therefore spectral distance and enstrophy over time with projection applied to every method.
Why not just add a conservation penalty to quantization-aware training? This is plausible, and it is compatible with our method rather than a substitute for it. A penalty changes how the remaining weights are trained but does not decide where the bits go. We run quantization-aware training with a penalty, both alone and combined with our allocation. If the penalty alone matches the combination at 2 bits, then allocation is unnecessary, and we will say so.
Why not just use TACQ with rollout error? This is the second ablation above. If rollout error matches physical diagnostics, the honest conclusion is that the horizon matters and not the physics, and the paper will report that outcome.
Proposed Experiments
The benchmarks are 1D Burgers', 2D Navier–Stokes and 2D Darcy flow from the FNO suite, together with 1D diffusion–sorption, following the PDE quantization study. If compute allows, we add a subset of the Exponax tasks used by PDE-Transformer to test whether the findings carry over to a Transformer operator, where wavenumber blocks are not explicit.
Four checks decide the outcome. The first tests whether physical diagnostics break before one-step error at 3 and 2 bits; it fails if the two degrade together. The second tests whether mass drift is linear in rollout length and collapses when only the zero-mode blocks are protected. The third tests whether our allocation beats both the transplanted TACQ baseline and the low-mode heuristic on Navier–Stokes spectral distance at a matched 2.5 average bits. The fourth tests whether the translation-equivariance control stays at numerical precision.
A positive result on the first three checks would establish that physical diagnostics are a better objective for compressing scientific models than error alone. A negative result on the third would still yield the first map of where quantization damage sits inside a neural operator, which is the gap that the language-model diagnosis left open.
References
- Task-Circuit Quantization: Leveraging Knowledge Localization and Interpretability for Compression
- Through a Compressed Lens: Investigating the Impact of Quantization on LLM Explainability and Interpretability
- Efficient Neural PDE-Solvers using Quantization Aware Training
- Adaptive Mesh Quantization for Neural PDE Solvers
- QuIP: 2-Bit Quantization of Large Language Models With Guarantees
- Understanding intermediate layers using linear classifier probes
- Fourier Neural Operator for Parametric Partial Differential Equations
- Deep learning in fluid dynamics
- CRVQ: Channel-Relaxed Vector Quantization for Extreme Compression of LLMs
- PDE-Transformer: Efficient and Versatile Transformers for Physics Simulations