Implicit Gradients Through the Projection Step of Manifold-Constrained Neural Simulators

Many physical systems have states that must satisfy exact rules: pendulum rods keep their length, power-grid nodes obey current balance, an orbiting body conserves energy. Neural models of these systems drift off the allowed set of states. The cleanest numerical fix is old and simple. Take an ordinary integration step, then pull the result back to the nearest allowed state. That pull-back is a small nonlinear optimization solved by a few Newton iterations at every time step. Nobody has worked out how to train through it well. If you backpropagate through every Newton iteration of every step, memory grows with both. If you skip the projection in the backward pass, the gradient is wrong by an amount nobody has measured. If the constraint itself is learned, as when a network discovers an unknown conserved quantity, the gradient with respect to the constraint has to come from somewhere.

We propose to treat each projection step as an optimization layer. Its gradient comes from implicit differentiation, which applies the implicit function theorem to the optimality conditions at the converged point, so no solver iterations need to be stored. The first payoff is engineering: exact gradients through hard post-step projections at constant memory per step, for both known and learned constraints. The second payoff is a testable prediction about when this matters. The exact backward pass differs from the cheap "just project the gradient onto the tangent space" shortcut only by a curvature term. That term is weighted by the Lagrange multiplier, and the multiplier is proportional to how far the step drifted off the manifold. So the shortcut should be fine for small steps on gently curved constraints. It should fail for large steps, high curvature, and learned constraints. We intend to confirm or refute this directly.

Core Claim

Let $\tilde u = \Phi^\theta_h(u_n)$ be one explicit Runge–Kutta step of a neural vector field with parameters $\theta$ and step size $h$. Let the allowed states be $\mathcal{M}_\phi = \{u : g_\phi(u) = 0\}$, where the $m$ constraint functions $g_\phi$ are either known or learned with parameters $\phi$. The projected update is the closest allowed point:

$$u_{n+1} = \arg\min_u \tfrac12 \|u - \tilde u\|^2 \quad \text{subject to} \quad g_\phi(u) = 0 .$$

At the solution, the optimality conditions are $u_{n+1} - \tilde u + J^\top \lambda = 0$ and $g_\phi(u_{n+1}) = 0$. Here $J = \nabla g_\phi(u_{n+1})$ is the constraint Jacobian and $\lambda \in \mathbb{R}^m$ is the multiplier. Differentiating these conditions gives one linear system per step:

$$\begin{bmatrix} I + \sum_i \lambda_i \nabla^2 g_{\phi,i} & J^\top \\ J & 0 \end{bmatrix} \begin{bmatrix} \delta u \\ \delta\lambda \end{bmatrix} = \begin{bmatrix} \delta \tilde u \\ -\partial_\phi g_\phi \, \delta\phi \end{bmatrix}.$$

The reverse-mode backward pass solves the transpose of this system once, at the converged point. It never touches the Newton iterates.

Two observations make this more than bookkeeping. First, when $\lambda = 0$ the system reduces exactly to the orthogonal tangent projector that Projected Neural Differential Equations for Learning Constrained Dynamics applies to the vector field. The cheap tangent-projector backward pass is therefore the zeroth-order truncation of the exact gradient. Second, $\|\lambda\|$ equals the distance from $\tilde u$ to the manifold, measured in the constraint metric. That distance is the integrator's local error, of order $h^{p+1}$ for a method of order $p$. We therefore predict that the per-step relative error of the tangent-projector shortcut scales like curvature times $h^{p+1}$. We also predict that the gradient with respect to $\phi$, which the shortcut cannot provide at all, is carried entirely by the right-hand side term $\partial_\phi g_\phi$.

Research Program

We will build this layer in JAX and attach it to three model families. The first is a Cartesian neural vector field with known constraints, the setting of PNDE. The second is a constrained Hamiltonian network, in which redundant coordinates and Lagrange multipliers replace angles, as in Simplifying Hamiltonian and Lagrangian Neural Networks via Explicit Constraints. The third is a model with a learned invariant network standing in for $g_\phi$, the setting of FINDE: Neural Differential Equations for Finding and Preserving Invariant Quantities.

Across steps we will store only step endpoints and multipliers, and recompute Runge–Kutta stages in the backward pass. This borrows the checkpointing logic of the Symplectic Adjoint Method for Exact Gradient of Neural ODE with Minimal Memory, which handles only unconstrained ODEs. The per-step linear system is $(d+m)$-dimensional and sparse in multi-body problems. We will solve it with the QR approach that Scalable Differentiable Physics for Learning and Control used for contact constraints. That paper differentiates hand-specified impact constraints in a fixed simulator; it neither learns vector fields or constraints nor studies the curvature term or solver tolerance. We supply those.

The analytical part has two pieces. First, we will derive the per-step and accumulated gradient error of the tangent-projector and identity backward passes as functions of $h$, constraint curvature, and horizon. Second, we will address the inexact-convergence problem that the research file identifies as uncharacterized (Gap 3). If Newton stops with residual $r = \|g_\phi(u_K)\|$, the implicit gradient evaluated at $u_K$ should be biased by roughly $r$ times the condition number of the system above, by standard first-order perturbation. We will test this bound and use it to set a per-step tolerance that keeps gradient bias below a target, rather than a fixed tolerance. Our approach differs from Efficient Riemannian Meta-Optimization by Implicit Differentiation in two ways. That work differentiates a discrete inner optimizer on a fixed manifold under a strictly convex inner loss. Here the manifold may move with $\phi$, the outer problem is a long nonconvex rollout, and the inner problem is a closest-point projection, which is well-posed only within the manifold's reach. We will monitor whether that reach is violated.

Baselines

The naive reference point is an unconstrained neural ODE trained by standard backpropagation, plus the same model with a soft penalty $\beta\|g(u)\|^2$. The penalty version is the stabilized neural ODE that PNDE compares against. Both get the same architecture, data, and hyperparameter search budget as our method, with $\beta$ swept over the same number of trials we spend tuning step size and tolerance. The reported number to beat on constraint satisfaction is from PNDE on the IEEE 14-bus and RTS-96 power grid. There the continuous tangent projection reaches constraint error $3.5 \times 10^{-11}$, against $0.5 \times 10^{-7}$ for the best-tuned penalty model (Projected Neural Differential Equations for Learning Constrained Dynamics).

For known constraints, the strongest existing method is PNDE itself: the vector field is projected at every solver stage and gradients come from ordinary autodiff. It is the strongest competitor because it already reaches near machine-precision constraint error. Our method therefore cannot win on that metric alone. It has to win on accuracy at matched wall-clock time, or match accuracy with larger steps and lower memory. For mechanical systems we also run the constrained Hamiltonian network of Simplifying Hamiltonian and Lagrangian Neural Networks via Explicit Constraints. It reports matching standard Hamiltonian networks with 100x less data and up to 260x higher trajectory accuracy on the gyroscope. We use its released benchmark systems and training budgets.

For learned constraints, the strongest baseline is continuous and discrete FINDE. The file reports that FINDE achieves higher valid prediction time than neural ODEs and Hamiltonian networks but gives no value: no reported result found on Lacuna for FINDE valid prediction time on the two-body, KdV, or FitzHugh–Nagumo benchmarks. We will therefore rerun it ourselves under equal compute.

Three ablations hold the forward pass fixed and change only the backward pass through the projection. The first unrolls the Newton iterations with autodiff. The second keeps only the tangent-projector term, setting $\lambda = 0$ in the system above. The third treats the projection as the identity. Our method beats these ablations if it matches the unrolled gradient to within solver tolerance at constant memory per step. It beats the shortcuts only if they measurably lose rollout accuracy or constraint learning in the regimes the theory predicts. It does not count as beating them if they tie everywhere.

Why not just…?

Why not just use PNDE's continuous tangent projection with ordinary autodiff? For known constraints at small step sizes this is a real threat, and it may tie us. Tangent projection keeps velocities on the manifold, but a discretized flow still leaves the manifold at order $h^{p+1}$ per step. PNDE's tiny errors rely on fine integration. Our experiments sweep step size on the damped N-pendulum and the power grid. The question is whether post-step projection with implicit gradients holds accuracy at steps where PNDE's error grows, at lower total cost. If PNDE wins across the sweep, the method's value narrows to the learned-constraint setting, and we will say so.

Why not just unroll the few Newton iterations? Newton converges quadratically, so unrolling three to five iterations costs only a constant-factor memory increase. At convergence it gives nearly the same gradient. The implicit approach must justify itself on two grounds. The first is memory over long horizons, where unrolling solvers is what exhausted 11GB in Scaling up and Stabilizing Differentiable Planning with Implicit Differentiation and drove the 4,479 MiB versus 20 MiB gap in the symplectic adjoint study. The second is behavior when Newton is truncated early, where the unrolled gradient is the gradient of a different map. We compare the two directly against a high-precision reference gradient while varying the iteration cap.

Why not just pass gradients through the tangent projector, or ignore the projection entirely? This is the cheapest option and the most serious alternative, because our own analysis says it is correct to zeroth order. The experiment that settles it measures gradient cosine similarity and final rollout error against the exact implicit gradient. We vary step size and constraint curvature, using longer or shorter pendulum links and the stiffer FPUT lattice. If the gap does not grow as curvature times $h^{p+1}$, our central prediction is wrong. For learned constraints the shortcut cannot update $\phi$ through the projection at all, so there it can only win if FINDE-style auxiliary losses suffice on their own.

Why not just use a penalty, or minimal coordinates? The file already contains evidence against both. The penalty model plateaus at $0.5\times10^{-7}$ constraint error against PNDE's $3.5\times10^{-11}$. Soft-penalized control learned non-physical parameters 16.85% of the time against 2.76% for fixed-point differentiation in DiLQR. Angle coordinates lost by up to 260x on the gyroscope in the constrained Hamiltonian network study. We include the penalty baseline only as a sanity anchor.

Proposed Experiments

We use the damped Cartesian N-pendulum, the IEEE 14-bus and RTS-96 power grid, and the FPUT lattice from PNDE. We add the gyroscope and chains of up to five pendulums from the constrained Hamiltonian network study, and the two-body problem, mass-spring system, and KdV soliton from FINDE, which serve as the learned-invariant tests. For each system we report relative state error over long rollouts, constraint or invariant error, valid prediction time, peak training memory, and wall-clock time per epoch. Every model gets the same data and tuning budget.

The decisive checks are three. First, plot gradient error of the tangent and identity shortcuts against step size and curvature, and test whether the slope matches the predicted $h^{p+1}$ scaling. Second, vary the Newton tolerance and test whether gradient bias tracks the residual-times-conditioning bound, and whether the adaptive tolerance holds bias down at lower cost than a fixed tight tolerance. Third, on the FINDE benchmarks, test whether exact implicit gradients with respect to the learned constraint make invariant discovery less fragile. We measure fragility as the spread of valid prediction time across random seeds, compared with continuous and discrete FINDE.

A clean negative result would also be informative. If the tangent shortcut matches the exact gradient everywhere, practitioners can stop paying for curvature terms.