Abstract
Standard machine learning solvers for partial differential equations (PDEs), such as Physics-Informed Neural Networks (PINNs), often struggle when equations involve highly irregular or "rough" forcing terms. These methods typically rely on pointwise evaluations of the PDE, which become mathematically ill-defined or numerically unstable in the presence of stochastic noise. This paper introduces a novel framework that replaces the standard $L^2$ loss with an empirical negative Sobolev norm ($H^{-s}$). By evaluating the PDE in a weak sense through integration against smooth test functions, the proposed method enables both kernel-based solvers and neural networks to accurately solve nonlinear Stochastic PDEs (SPDEs). We provide theoretical convergence guarantees even in cases of model misspecification, where the true solution lies outside the hypothesis space of the solver.
Introduction
Machine learning has revolutionized the numerical solution of PDEs, with Physics-Informed Neural Networks (PINNs) and Gaussian Process (GP) solvers providing flexible, mesh-free alternatives to traditional finite element methods. However, a significant limitation of these approaches is their reliance on "strong form" residuals. Typically, these models minimize an empirical $L^2$ loss by checking if the PDE holds at a finite set of discrete collocation points.
This approach assumes the existence of a classical, smooth solution. In many physical and biological systems, however, PDEs are driven by stochastic forcing terms like white noise. These Stochastic PDEs (SPDEs) result in solutions that are jagged and non-differentiable in the classical sense. Attempting to apply pointwise evaluation to such systems causes the loss function to diverge or become highly unstable. Furthermore, existing theoretical frameworks for GP solvers often require the solution to reside within the Reproducing Kernel Hilbert Space (RKHS) of the chosen kernel—a condition that is fundamentally violated by the irregular nature of SPDE solutions.
Methodology: The Negative Sobolev Norm
To address the instability of pointwise evaluations, this paper proposes a shift from the strong form of the PDE to a weak formulation. The core innovation is the use of an empirical negative Sobolev norm ($H^{-s}$) as the objective function.
Instead of requiring the PDE to be satisfied at specific points, the model is required to satisfy the PDE when integrated against a set of smooth test functions. This integration acts as a smoothing operator, filtering the high-frequency noise of the rough forcing term and making the optimization problem mathematically tractable.
The authors implement this approach across two distinct architectures:
- Misspecified Kernel Methods: The authors formulate an optimal recovery problem using an "oversmoothing" GP kernel. Crucially, they prove that even when the kernel is misspecified—meaning the rough solution does not live in the smooth RKHS—minimizing the weak loss via a Gauss-Newton algorithm still ensures asymptotic convergence to the true solution.
- NeS-PINN (Negative Sobolev PINN): The same weak loss function is adapted for neural networks. By replacing the standard PINN loss with the $H^{-s}$ norm, the network can learn to approximate solutions to SPDEs that would otherwise be impossible to train using standard methods.
Experimental Results
The methodology was evaluated on several benchmark problems involving manufactured rough forcing terms generated via truncated Fourier series with Gaussian coefficients.
Elliptic PDEs
In a 1D linear elliptic PDE task, the kernel method using the $H^{-1}$ loss achieved a relative $L^2$ error of $3.95 \times 10^{-5}$, whereas the standard pointwise loss failed completely with an error of $1.68$. Similarly, the NeS-PINN achieved an error of $0.0200$, compared to $1.49$ for a standard PINN. In 2D semilinear cases, NeS-PINN maintained high accuracy (error of $0.0203$) while the standard PINN diverged (error of $3.40$).
Time-Dependent SPDEs
The framework was also applied to dynamic systems, specifically the Stochastic Heat Equation and the nonlinear Stochastic Allen-Cahn Equation driven by space-time white noise. The kernel method successfully tracked the fine-mesh ground truth solutions for both, achieving errors of $6.84 \times 10^{-3}$ and $2.07 \times 10^{-2}$ respectively, demonstrating the robustness of the weak loss in capturing complex, time-evolving stochastic behavior.
Discussion and Insights
A key theoretical takeaway is that misspecification is not fatal. The proof that oversmoothing kernels can still achieve convergence provided the correct weak loss is used opens the door for using highly regular kernels on irregular data.
However, the success of the method depends heavily on the choice of the Sobolev parameter $s$. This creates a "Goldilocks" problem:
- If $s$ is too small, the loss remains sensitive to noise and training is unstable.
- If $s$ is too large, the model oversmooths the solution, losing critical high-frequency details.
Conclusion and Future Work
By moving from pointwise residuals to a negative Sobolev norm, this work provides a mathematically rigorous and numerically stable path for solving roughly forced nonlinear PDEs. Future research directions include the development of "curriculum learning" strategies to automatically tune the parameter $s$, extending the framework to handle boundary conditions in a weak "trace sense," and applying these methods to singular SPDEs like the Kardar-Parisi-Zhang (KPZ) equation, which require advanced renormalization techniques.
Summary updated