Theoretical Convergence of Physics-Informed Learning

This direction is split into other directions: linear operator restriction, generalization to nonlinear PDEs. Its last version stays readable and citable here.

Version 1 of 1435 papers465 conceptsfrozen 2026-10-04

Physics-Informed Machine Learning (PIML) has emerged as a powerful paradigm for solving partial differential equations (PDEs) by embedding physical laws directly into neural network loss functions. However, a persistent gap exists between the empirical success of these models and our theoretical understanding of why they converge. While researchers have established rigorous guarantees for linear, elliptic systems, extending these proofs to the nonlinear and hyperbolic equations that govern complex fluid dynamics and wave propagation remains the primary frontier of the field.

The Fast Track of Physical Priors

The central intuition behind physics-informed learning is that physical knowledge acts as a "fast-track" for optimization. In standard machine learning, we often assume data is independent and identically distributed (i.i.d.). When dealing with physical systems, data is frequently dependent—for example, measurements taken along a single trajectory.

Anna Scampicchio, James Anderson, and their colleagues Scampicchio et al. (2025) recently proved that well-aligned physical priors allow models to achieve "fast" convergence rates ($O(1/T)$) even with dependent data. Essentially, the physics acts as a variance stabilizer, smoothing out noise and allowing the model to "overfit" to noisy measurements in a way that still generalizes to the underlying true solution—a phenomenon known as benign overfitting Wong et al. (2024).

The Conditioning Problem

If physics helps so much, why are Physics-Informed Neural Networks (PINNs) notoriously difficult to train? The answer lies in the "conditioning" of the differential operator. Tim De Ryck has shown that the training speed of a PINN is governed by the Hermitian square of the PDE's operator ($D^*D$). When this operator is ill-conditioned, the loss landscape becomes rugged, leading to slow convergence or total failure De Ryck (2024).

To fix this, researchers borrow a classic tool from numerical analysis: operator preconditioning. By transforming the operator to improve its spectral properties, De Ryck et al. (2023) demonstrated that we can turn failing training processes into fast, exponentially converging ones. Similarly, Felix Dangel introduced second-order optimization frameworks like KFAC to handle the curvature of these complex loss functions Dangel (2024).

The Linear Wall

Most of our rigorous mathematical "safety nets"—such as error bounds and convergence proofs—currently hit a wall at linear operators.

  • Neural Tangent Kernels (NTK): Sifan Wang and Paris Perdikaris used NTK theory to explain why PINNs struggle with high-frequency components, but their analysis is largely restricted to linear PDEs Wang et al. (2020).
  • Error Analysis: Tim De Ryck and Siddhartha Mishra proved that PINNs can overcome the "curse of dimensionality" for Kolmogorov equations, but these are linear parabolic systems De Ryck & Mishra (2021).
  • Bayesian Inversion: Theoretical frameworks for posterior contraction rates—how quickly our uncertainty shrinks as we get more data—often rely on the assumption of a linear forward operator Agapiou et al. (2013).

Moving Toward Nonlinearity

The research community is now focused on breaking through this linear restriction. One strategy is iterative linearization, where a nonlinear problem is broken down into a sequence of linear ones during training. Lizuo Liu and Wei Cai applied this to the Navier-Stokes equations, achieving faster convergence by decoupling nonlinear convection terms Liu & Cai (2021).

Another approach involves architectural priors. Models like BENO (Boundary-Embedded Neural Operators) mimic the mathematical structure of Green’s functions to handle complex geometries and inhomogeneous boundary conditions Wang et al. (2024). Meanwhile, Jiachen Yang and colleagues are using reinforcement learning to enable "anticipatory refinement," where the mesh itself learns to adapt to nonlinear features like turbulence before they even arrive Yang et al. (2022).

The ultimate goal is a unified theory that provides the same level of mathematical guarantee for a nonlinear fluid simulation as we currently have for a simple heat equation. As Weiye Gan and Zuoqiang Shi note, understanding how complex differential operators affect the stability of the network's kernel is the key to unlocking the next generation of scientific AI Gan et al. (2025).

Papers in this version

Showing 60 of 435 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_theoretical_convergence_of_physics_infor_227d91d2,
  title = {Theoretical Convergence of Physics-Informed Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/theoretical-convergence-of-physics-informed-learning/rdr_227d91d2940d4521b4b86f350b567a44},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_theoretical_convergence_of_physics_infor_227d91d2_v1,
  title = {Theoretical Convergence of Physics-Informed Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/theoretical-convergence-of-physics-informed-learning/rdr_227d91d2940d4521b4b86f350b567a44/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}