Neural networks now stand in for expensive physics simulators, predicting the next state of a fluid or atmosphere from the current one. Scientists want to know why such a surrogate predicts what it does, and the usual tool is a gradient saliency map: the derivative of an output with respect to each input location, shown as a heatmap of "what the prediction depended on." In image classification nobody knows what the correct saliency map is. That is why explanation stability research argues about whether a map that jitters under tiny input changes is wrong or merely honest. In simulation, though, a correct answer exists. The numerical solver that generated the training data has its own derivative, which says exactly how the true next state depends on every grid cell of the current one. A faithful saliency map for a one-step surrogate should reproduce that derivative, and a stable one should change exactly as much as that derivative changes.
We propose to use this ground truth to measure, and then fix, saliency in neural PDE surrogates. There are three steps. First, we build a benchmark in which each surrogate's input-gradient saliency is scored against the solver's derivative, computed either by automatic differentiation or by two solver calls. Second, we use this benchmark to settle an open question: do physical-consistency constraints (conservation correctors, divergence-free architectures, physics penalty losses) make saliency more trustworthy, or does the extra differentiation they require make it noisier? Third, we test a cheap training signal. It asks the surrogate's directional derivatives to match the solver's along random physically plausible perturbations. Our hypothesis is that this aligns saliency with physics directly, which conservation constraints do not.
Core Claim
Our central claim is that, for learned simulators, explanation stability should be judged against the solver's Jacobian rather than against the model's own inputs or internal representations. This resolves the faithfulness-versus-smoothness tension that runs through the stability literature. Let $F_\theta$ be a surrogate mapping state $u_t$ to $u_{t+\Delta t}$, and let $S$ be the reference solver. Let $J_F(u)$ and $J_S(u)$ be their Jacobians, the matrices of partial derivatives of every output cell with respect to every input cell. The saliency map for output location $x$ is row $x$ of $J_F$, and its ground truth is row $x$ of $J_S$.
We define physics-relative stability for a small, physically admissible perturbation $\delta$ of the input state as
$$\mathrm{PRS}(u,\delta) = \frac{\lVert J_F(u+\delta) - J_F(u)\rVert}{\lVert J_S(u+\delta) - J_S(u)\rVert}.$$
A value near 1 means the explanation changes exactly as much as the physics does. Values well above 1 indicate spurious jitter of the kind Zifan Wang et al. attribute to jagged decision surfaces. Values well below 1 indicate over-smoothing that erases real sensitivity, for example the abrupt change in dependence as a Burgers shock passes.
This extends Rethinking Stability for Attribution-based Explanations. That paper normalizes attribution change by change in the model's inputs, representations, or outputs, and correctly argues that explanations should move when reasoning moves. Its normalizers are all internal to the model, however, so a network tracking an unphysical shortcut can look perfectly stable. Our normalizer is external to the model and physical.
The proposed intervention is a Jacobian-vector-product matching loss. A Jacobian-vector product is the directional derivative along a chosen input perturbation $v$. The loss is
$$\mathcal{L}_{\mathrm{JVP}} = \mathbb{E}_{u,\,v \sim P_{\mathrm{phys}}}\big\lVert J_F(u)\,v - J_S(u)\,v \big\rVert^2,$$
where $P_{\mathrm{phys}}$ draws perturbations that respect the governing constraints. For incompressible flow these are divergence-free and band-limited to resolved scales. For shallow water they are mass-preserving. Each term costs one forward-mode derivative of the surrogate plus one solver derivative, or a finite difference $(S(u+\varepsilon v)-S(u))/\varepsilon$ when the solver is a black box.
Research Program
The work has three parts, and each uses a different direction as its lens.
Measurement. Gradient saliency is the object of study and the solver Jacobian is the yardstick. We will implement or wrap differentiable reference solvers for four PDEBench problems from PDEBench:
- 1D advection, where the true Jacobian is an exact shift and so a sanity check;
- 1D Burgers, which has shocks;
- 2D incompressible Navier-Stokes;
- 2D shallow water.
We will add Kuramoto-Sivashinsky from Learned Simulators for Turbulence as a chaotic case. On held-out states we compute three quantities from sampled rows of the Jacobian:
- Jacobian faithfulness: relative error and cosine similarity between surrogate and solver rows.
- Out-of-cone saliency mass: the fraction of saliency falling outside the solver's domain of dependence, the region information can physically reach in one step.
- Physics-relative stability, as defined above, reported separately in smooth regions and near discontinuities.
Diagnosis. Physical inconsistency is the regime. We train standard surrogates (Fourier Neural Operator, U-Net, and dilated ResNet) and then add each consistency mechanism from the literature:
- a soft PDE-residual penalty;
- a global conservation projection in the style of the ACE2 corrector (Brian Henn et al.);
- an exact divergence-free parameterization from Neural Conservation Laws for the incompressible case.
This directly tests the tension the literature leaves open. Analyzing Explainer Robustness via Probabilistic Lipschitzness proves that explainer robustness is lower-bounded by the smoothness of the prediction function, which suggests constrained, smoother models should explain more stably. The divergence-free construction instead builds outputs from Jacobians of an inner network. Saliency then passes through second derivatives of that network, which could amplify jaggedness. We expect conservation mechanisms to cut conservation error, but only weakly affect Jacobian faithfulness. A single global integral constraint pins down one direction of the Jacobian, not its local structure.
Intervention. Explanation stability is the target property. We train with $\mathcal{L}_{\mathrm{JVP}}$ added to the usual one-step loss, at a matched compute budget, and ask whether it moves physics-relative stability toward 1 in both smooth and shock regions.
A secondary diagnostic concerns shortcuts. Following the controlled-distractor design of Seeing Through Distractions, we plant a shortcut: an auxiliary boundary channel correlated with inflow in training but not at test. We then ask whether out-of-cone saliency mass flags the shortcut-using model at the cost of a single backward pass. This is a far cheaper test-time alternative to the inverse-Hessian influence computations of PINNfluence. That paper attributes predictions to training points rather than input locations, so it answers a different question.
Baselines
Unmodified surrogates with vanilla gradient saliency. The naive reference is each architecture trained with plain one-step mean squared error and explained by its raw input gradient. This is what a practitioner gets by default, and every other method must beat it on Jacobian faithfulness without losing more than a small, pre-set margin of one-step error. PDEBench reports that the Fourier Neural Operator outperforms U-Nets and PINNs on forward prediction, but the file lists no numerical table entries. We therefore report no reported result found on Lacuna for PDEBench MSE or cRMSE values and will reproduce these baselines ourselves using PDEBench's data splits.
SmoothGrad applied post hoc. SmoothGrad averages gradients over noise-perturbed copies of the input. It is the strongest existing stabilizer: Rethinking Stability for Attribution-based Explanations found it the most stable explainer, with an average stability improvement of 12.7% on German Credit, COMPAS, and Adult. We run it on the same unmodified surrogates. Its noise scale and sample count are tuned on a validation split to maximize Jacobian faithfulness, which gives it the most favorable possible setting. Beating it means lower Jacobian error overall and physics-relative stability closer to 1 near shocks. There we predict SmoothGrad falls well below 1 because it averages across the discontinuity.
Lipschitz regularization. This is training with a smoothness penalty, as analyzed by Khan et al.. It is the theoretically motivated competitor, since their bound says smoothness alone should buy explainer robustness. We sweep its strength over the same number of configurations as our loss weight.
Fourier gradient prior adapted to 2D fields. This is the training penalty on high-frequency content of input gradients from Fourier-transform-based attribution priors. That paper reports roughly 40% higher attribution Jaccard similarity across seeds on SPI1 genomics models. It is the closest existing attribution-level training signal. We give it the same tuning budget, with its frequency cutoff chosen per problem.
Physical-consistency mechanisms. These are the soft PDE-residual penalty, the ACE2-style conservation corrector, and the Neural Conservation Laws architecture. They are the natural objection that "physics-consistent models already explain well." PreDiff shows how much such alignment can improve physical metrics, reducing N-body MNIST energy MSE from 0.0226 to 0.0039, but it reports no saliency measure. We use the same architectures, data, and optimizer budget for each.
Ablations of our loss. Two ablations isolate the key ingredient. The first draws probe directions $v$ from isotropic Gaussian noise instead of physically admissible perturbations; this tests whether the physical probe distribution matters. The second replaces solver derivatives with derivatives of a second independently trained surrogate; this tests whether the solver reference is what helps, rather than derivative matching as a generic regularizer.
Why not just…?
Why not just apply SmoothGrad to the surrogate? SmoothGrad is cheap and was the most stable explainer in the tabular study above, so it is a real competitor for the "stability" half of the problem. It has two weaknesses here. Its isotropic noise leaves the space of physical states, for example by adding divergence. It also can only average the model's existing gradients, so it cannot repair a model whose Jacobian is wrong; Khan et al. show explainer robustness is bounded by the model's own geometry. Our benchmark settles this directly. If tuned SmoothGrad matches the Jacobian-matched model on faithfulness and keeps physics-relative stability near 1 at shocks, the training intervention is unnecessary and we will say so.
Why not just enforce conservation, as ACE2 or Neural Conservation Laws do? Conservation fixes drift in integrated quantities, as ACE2 shows over century-long rollouts. However, conservation is a constraint on sums over the domain. A model can conserve mass exactly while drawing its prediction for a cell from the wrong neighbors. The diagnosis experiment measures whether these mechanisms improve Jacobian faithfulness at all. If they turn out to be sufficient, that is itself a useful and publishable finding, and the loss becomes optional.
Why not just train longer on more trajectories, or use a bigger model? Scale is the most serious threat. A perfectly fit surrogate would have the solver's Jacobian automatically, although the WEATHER-5K results in Han et al. suggest inductive bias beats parameter count in weather. We will test this by plotting Jacobian faithfulness against training-set size and model width for the unmodified baseline, then comparing it with the Jacobian-matched model at equal total compute. The extra solver calls are counted as data. If the scaling curve reaches the same faithfulness at equal compute, our claim is limited to data-scarce regimes.
Proposed Experiments and Decisive Checks
All results are averaged over five seeds, and differences are reported only when they exceed seed-to-seed spread. Four outcomes would decide the proposal.
- Does Jacobian error at one step predict long-horizon failure? We test whether one-step Jacobian error across all trained models predicts long-rollout drift in PDEBench's conservation error (cRMSE) and spectral error (fRMSE). A strong rank correlation would make solver-referenced saliency a cheap early warning for rollout failure.
- Do hard constraints help or hurt saliency? The Neural Conservation Laws model's physics-relative stability, relative to the unconstrained model, answers the question of whether hard constraints stabilize or destabilize saliency.
- Does Jacobian matching work? It succeeds if it beats tuned SmoothGrad and the Fourier prior on Jacobian faithfulness, keeps physics-relative stability within a pre-registered band around 1 in both smooth and shock regions, and stays within the one-step error margin. If the isotropic-probe ablation performs as well, the physical probe distribution is not the active ingredient and we will report that.
- Does out-of-cone saliency catch shortcuts? In the planted-shortcut experiment, out-of-cone mass should separate shortcut-using models from clean ones at test time using a single backward pass.
The file contains no prior number for saliency faithfulness against a solver Jacobian; there is no reported result found on Lacuna for that quantity. Establishing the first such numbers is part of the contribution.
References
- Smoothed Geometry for Robust Attribution
- Rethinking Stability for Attribution-based Explanations
- PDEBench: An Extensive Benchmark for Scientific Machine Learning
- Learned Simulators for Turbulence
- ACE2: accurately learning subseasonal to decadal atmospheric variability and forced responses
- Neural Conservation Laws: A Divergence-Free Perspective
- Analyzing Explainer Robustness via Probabilistic Lipschitzness of Prediction Functions
- Seeing Through Distractions: Stable Attribution via the Core
- PINNfluence: Interpreting PINNs through Influence Functions
- Fourier-transform-based attribution priors improve the interpretability and stability of deep learning models for genomics
- PreDiff: Precipitation Nowcasting with Latent Diffusion Models
- Benchmarking Physics-Informed Time-Series Models for Operational Global Station Weather Forecasting