Physics-Informed Machine Learning (PIML) has emerged as a powerful paradigm for solving partial differential equations (PDEs) by embedding physical laws directly into neural network loss functions. However, a persistent gap exists between the empirical success of these models and our theoretical understanding of why they converge. While researchers have established rigorous guarantees for linear, elliptic systems, extending these proofs to the nonlinear and hyperbolic equations that govern complex fluid dynamics and wave propagation remains the primary frontier of the field.
The Fast Track of Physical Priors
The central intuition behind physics-informed learning is that physical knowledge acts as a "fast-track" for optimization. In standard machine learning, we often assume data is independent and identically distributed (i.i.d.). When dealing with physical systems, data is frequently dependent—for example, measurements taken along a single trajectory.
Anna Scampicchio, James Anderson, and their colleagues Scampicchio et al. (2025) recently proved that well-aligned physical priors allow models to achieve "fast" convergence rates ($O(1/T)$) even with dependent data. Essentially, the physics acts as a variance stabilizer, smoothing out noise and allowing the model to "overfit" to noisy measurements in a way that still generalizes to the underlying true solution—a phenomenon known as benign overfitting Wong et al. (2024).
The Conditioning Problem
If physics helps so much, why are Physics-Informed Neural Networks (PINNs) notoriously difficult to train? The answer lies in the "conditioning" of the differential operator. Tim De Ryck has shown that the training speed of a PINN is governed by the Hermitian square of the PDE's operator ($D^*D$). When this operator is ill-conditioned, the loss landscape becomes rugged, leading to slow convergence or total failure De Ryck (2024).
To fix this, researchers borrow a classic tool from numerical analysis: operator preconditioning. By transforming the operator to improve its spectral properties, De Ryck et al. (2023) demonstrated that we can turn failing training processes into fast, exponentially converging ones. Similarly, Felix Dangel introduced second-order optimization frameworks like KFAC to handle the curvature of these complex loss functions Dangel (2024).
The Linear Wall
Most of our rigorous mathematical "safety nets"—such as error bounds and convergence proofs—currently hit a wall at linear operators.
- Neural Tangent Kernels (NTK): Sifan Wang and Paris Perdikaris used NTK theory to explain why PINNs struggle with high-frequency components, but their analysis is largely restricted to linear PDEs Wang et al. (2020).
- Error Analysis: Tim De Ryck and Siddhartha Mishra proved that PINNs can overcome the "curse of dimensionality" for Kolmogorov equations, but these are linear parabolic systems De Ryck & Mishra (2021).
- Bayesian Inversion: Theoretical frameworks for posterior contraction rates—how quickly our uncertainty shrinks as we get more data—often rely on the assumption of a linear forward operator Agapiou et al. (2013).
Moving Toward Nonlinearity
The research community is now focused on breaking through this linear restriction. One strategy is iterative linearization, where a nonlinear problem is broken down into a sequence of linear ones during training. Lizuo Liu and Wei Cai applied this to the Navier-Stokes equations, achieving faster convergence by decoupling nonlinear convection terms Liu & Cai (2021).
Another approach involves architectural priors. Models like BENO (Boundary-Embedded Neural Operators) mimic the mathematical structure of Green’s functions to handle complex geometries and inhomogeneous boundary conditions Wang et al. (2024). Meanwhile, Jiachen Yang and colleagues are using reinforcement learning to enable "anticipatory refinement," where the mesh itself learns to adapt to nonlinear features like turbulence before they even arrive Yang et al. (2022).
The ultimate goal is a unified theory that provides the same level of mathematical guarantee for a nonlinear fluid simulation as we currently have for a simple heat equation. As Weiye Gan and Zuoqiang Shi note, understanding how complex differential operators affect the stability of the network's kernel is the key to unlocking the next generation of scientific AI Gan et al. (2025).