Multiplier, Projection, or Parameterization: When Does the Way a Learned Solver Enforces Conservation Matter?

Neural surrogates for physical simulation tend to lose mass, momentum, or energy over long rollouts. Several recent papers fix this with hard constraints that make the network obey a conservation law exactly. Each paper uses a different mechanism. One rescales the network's output so a global total matches. One builds Newton's third law into the convolution filters. One forces the network to update cells only by exchanging fluxes between neighbors. The classical numerical literature adds a fourth option: compute Lagrange multipliers, which are constraint forces chosen at each step so the state stays on the allowed set. That literature argues the fourth option is the physically correct one and that "step freely, then project back" breaks energy and momentum conservation. Nobody has tested this argument for learned solvers. Each method is evaluated on its own benchmark, with its own backbone, against weak baselines.

This proposal runs that test under controlled conditions. The core claim has two parts and can be falsified. For constraints that are linear in the state, such as cell-by-cell mass balance, the hard-constraint mechanisms should be nearly equivalent in accuracy and differ mainly in cost and in the boundary conditions they can express. For nonlinear geometric constraints, such as fluid confined to a sphere, enforcing the constraint as a force inside the time step should produce measurably less energy and angular-momentum drift than projecting after the step. If either half fails, the community learns which of the current design choices actually matter.

Core Claim

The argument rests on a simple fact. Suppose a model proposes a state $\tilde u$ and the constraint is linear, $Au = b$. Here $A$ might be the discrete divergence or the finite-volume balance for each cell, and $b$ the prescribed sources and boundary fluxes. The Lagrange-multiplier solution then coincides exactly with the Euclidean projection:

$$u = \tilde u - A^\top \lambda, \qquad \lambda = (AA^\top)^{-1}(A\tilde u - b),$$

where $\lambda$ is the vector of multipliers, one per constraint. In incompressible flow, $\lambda$ is the discrete pressure, the same pressure that Variational methods, multisymplectic geometry and continuum mechanics obtains as a pointwise multiplier field in the continuum. So for linear constraints, "multiplier versus projection" is a false choice. The real differences are three:

  1. Whether the constraint is local (one row per cell) or global (one row for the whole domain).
  2. Whether $b$ can carry nonzero boundary flux.
  3. What it costs to apply $(AA^\top)^{-1}$, compared with a parameterization that satisfies the constraint by construction, such as an antisymmetric flux update.

For a nonlinear constraint $g(x)=0$, such as $|x|^2-R^2=0$, the two operations separate. Renormalizing after a step applies the correction at the wrong point and does no work-free accounting. A constrained step instead solves for the multiplier so the constraint force stays normal to the manifold throughout the step. This is the mechanism used in the Hamiltonian particle-mesh method for spherical shallow water and in discrete constrained variational integrators on Riemannian manifolds. The classical claim that projection causes dissipation should therefore appear here and nowhere else. This is where the proposal expects the multiplier mechanism to earn its implicit-solve cost.

The new contribution is not a new architecture. It is a controlled, cost-aware separation of these mechanisms on shared backbones and shared benchmarks. It also introduces a local conservation metric that exposes failures the global metrics hide.

Prior Work and Its Role Here

Three papers supply the hard-constraint mechanisms we compare, and each motivates the study by leaving the comparison open.

PINN-Proj projects a physics-informed network's output so the spatial integral of the solution matches its reference value. It cuts conservation error by three to four orders of magnitude on 1D PDEBench problems. However, its constraint is a single global row of $A$. A deficit in one region can therefore be cancelled by a surplus elsewhere. The authors also state that it assumes no net boundary flux.

ASCC makes particle interaction kernels antisymmetric, which guarantees linear momentum conservation in learned particle fluids. It does not guarantee angular momentum.

FluxGNN proves that a graph network conserves locally exactly when its edge messages are antisymmetric and its node updates are linear. It reports local errors of $10^{-5}$ to $10^{-8}$. It achieves this by parameterization rather than by multipliers, and it shows accumulating rollout error over time.

Our study does not compete with these papers. It asks which of their ingredients is responsible for the gains they report.

Numerical PDE Solvers Outperform Neural PDE Solvers constrains the study. It shows that a differentiable finite-difference solver with learned coefficients beats neural operators by one to two orders of magnitude in error. Any learned constrained model must therefore be measured against a structured numerical baseline, not only against unconstrained networks.

PDEBench supplies data and global metrics: cRMSE for conservation, bRMSE for boundaries, and fRMSE for frequency content. It also acknowledges that its domains are rectangular. This is why we build a spherical track.

The validation-gap papers frame our methodology. They show proven rates that do not survive real constants, as in Nibble local clustering, and costs that go unmeasured, as in condition-number-independent gradient access. Both patterns explain why we report wall-clock time next to every accuracy number. The constrained variational integrator paper itself admits that its implicit steps raise per-iteration cost and that it was tested only on synthetic eigenvalue and Procrustes problems.

Research Program

We fix the backbone within each track and change only the enforcement layer. This gives five arms:

  1. Unconstrained. No conservation enforcement.
  2. Soft penalty. The conservation residual is added to the loss.
  3. Global projection. PINN-Proj style, with one row of $A$.
  4. Local multiplier projection. Uses $A$ with one row per cell and a $b$ that includes the observed boundary fluxes. It is solved with a conjugate-gradient or multigrid inner loop, and the solver is differentiated through during training.
  5. Antisymmetric flux parameterization. FluxGNN or ASCC style.

On the nonlinear track, arm 4 becomes a constrained time step that solves for the normal multiplier at every step. Arm 3 becomes renormalization after an unconstrained step.

We also introduce a local conservation residual. For each cell, it measures the discrete balance between the change in content and the net face flux, and it reports the maximum and the 99th percentile over cells and time. We report this residual alongside PDEBench's cRMSE. An expected side result is a demonstration that global projection can drive cRMSE to $10^{-6}$ while leaving local residuals orders of magnitude larger. This would be direct evidence that summary conservation metrics can mislead.

Beyond accuracy, we track energy drift, angular momentum drift on rotating problems, and the rollout length before error exceeds a threshold or the simulation returns NaN. We also record wall-clock time per step on fixed hardware across at least three spatial resolutions. From these we plot an accuracy-versus-cost Pareto frontier for each arm.

Baselines

The naive reference point is the unconstrained backbone trained on the same data. It sets the floor and shows how much any constraint helps. We regard any hard-constraint arm that fails to beat it on long-horizon rollout error as a negative result worth reporting.

The soft-penalty arm is a loss-weighted conservation residual, tuned over the same grid of penalty weights that the other arms receive for their hyperparameters. This is the default most practitioners use. PINN-Proj reports that hard projection beats it by roughly three orders of magnitude in conservation error on Burgers' and advection. Our arms beat it if they reproduce that gap in local residual, and not only in global error, on 2D problems.

The strongest existing hard-constraint method for the linear track is antisymmetric flux parameterization, implemented following FluxGNN. FluxGNN reports local conservation errors between $10^{-5}$ and $10^{-8}$, and its standard-GNN baselines diverge on out-of-domain "Taller" meshes. It is the strongest competitor because it achieves local conservation with no inner solve. We run it with an identical encoder width, training budget, and tuning trials. The local multiplier arm beats it only if it matches FluxGNN's local residual and also improves rollout error or open-boundary accuracy at equal or lower wall-clock cost. Our prior expectation is that it will not. A tie on the linear track would confirm the first half of the core claim.

For the global-projection arm we reimplement PINN-Proj. On KdV it reports a conservation error of $1.65\times10^{-6}$, against $5.55\times10^{-1}$ for a vanilla PINN. We first reproduce that number on its own 1D setting before carrying the arm into 2D.

For particle problems, ASCC is the reference. It reports a 7.22× EMD reduction over GNS on WBC-SPH, with 29% of the weights and 2.79× faster inference.

The structured numerical baseline is DeepFDM on rectangular problems. It reports MSE one to two orders of magnitude below FNO and relative out-of-distribution error below 1%. Any learned arm that does not approach it on rectangular grids carries the burden of justifying itself through mesh flexibility.

The ablation that isolates our key ingredient is the local-versus-global split with everything else fixed: the same multiplier solve with one row per cell versus one row per domain. A second ablation compares, on the sphere, a constraint force applied within the step against renormalization after the step, with identical learned tangent forces.

Why not just…?

Why not just use antisymmetric flux parameterization everywhere, since it is local, exact, and solver-free? For linear conservation laws, this is the real threat, and we expect it to win on cost. The linear track is designed to show this directly rather than argue around it. It does not cover two cases. The first is state constraints that are not flux balances, chiefly divergence-free velocity in incompressible flow, where pressure enters as a multiplier. The second is nonlinear manifold constraints, where FluxGNN and ASCC provide no mechanism at all. ASCC's own authors note that it leaves angular momentum unguaranteed. The sphere track tests whether that gap costs anything measurable.

Why not just parameterize velocity as the curl of a learned stream function, which makes it divergence-free by construction? In 2D on simple domains this is exact and cheap. We include it as an extra arm on the incompressible split. It becomes awkward with open inflow and outflow boundaries and on curved surfaces, where the stream function must satisfy compatibility conditions. The open-boundary experiment will show whether this awkwardness causes real accuracy loss.

Why not just renormalize onto the sphere after each step? Renormalization costs one division and keeps particles exactly on $S^2$. The classical claim that it causes artificial dissipation comes from numerical analysis of exact dynamics. A learned force model's own error may swamp the effect. This is exactly what the sphere track measures. If energy and angular-momentum drift under renormalization are statistically indistinguishable from drift under the constrained step, the second half of our claim is falsified. We would then report that the multiplier's extra cost buys nothing at learned-model accuracy.

Why not just use a differentiable numerical solver like DeepFDM? On rectangular grids it may simply be best, and we include it so that this is visible. Its authors note that changing geometry or boundary conditions requires re-deriving stencils. Our sphere and irregular-mesh settings are where that limitation would apply.

Proposed Experiments

Linear track. We use PDEBench 1D advection, Burgers', and KdV to reproduce the PINN-Proj results. We then move to PDEBench's 2D shallow-water and incompressible Navier-Stokes splits, and to the FluxGNN rotation, scaling, and Taller mesh suite for out-of-domain geometry. We also build an open-boundary variant by cropping PDEBench domains so that mass flows across the cut, with the observed face fluxes supplied as $b$. Global projection cannot represent this case without modification.

Nonlinear track. We generate spherical shallow-water trajectories with a Hamiltonian particle-mesh solver following the HPM spherical method. We also generate flocking trajectories on $S^2$ following Cucker–Smale on a sphere, which conserve angular momentum. For both, we train a shared tangent-force network and compare the two constraint-enforcement steps.

Decisive checks. There are four:

  1. On the linear track, local multiplier projection and flux parameterization should agree in local residual to within an order of magnitude. Flux parameterization should be cheaper per step. A large accuracy gap in either direction would contradict the equivalence argument and need explanation.
  2. Global projection should show low cRMSE together with a high local residual on 2D problems. It should also fail on the open-boundary variant.
  3. On the sphere, constrained steps should show lower energy and angular-momentum drift than renormalization over long rollouts, by a margin larger than the seed-to-seed spread.
  4. The wall-clock Pareto plot across resolutions should show whether the implicit solve's cost grows faster than the backbone's cost. That measurement was missing from the variational integrator work.

Every comparison uses at least five seeds with matched tuning budgets. Negative outcomes are reported with the same weight as positive ones.