Newton-PINet: A fast physics-informed neural network with Newton linearization for meta-learning nonlinear PDEs

Yuchen Fan, Chang Wei, Pao-Hsiung Chiu, Chin Chun Ooi, Heyang Wang, Jian Cheng WongarXiv 2026 · arXiv

TL;DR

Newton-PINet is a novel meta-learning framework that drastically accelerates the training of Physics-Informed Neural Networks (PINNs) for nonlinear partial differential equations (PDEs). By combining a skip-connected architecture with a two-stage learning strategy and Newton linearization, it achieves up to three orders of magnitude better accuracy than state-of-the-art solvers while requiring 16× fewer training tasks and reducing training time from hours to just five minutes.

What Problem Does This Paper Address?

While Scientific Machine Learning (SciML) has revolutionized how we solve PDEs, two major hurdles remain:

  1. Data Hunger: Data-driven operator learners (like DeepONet or FNO) require massive labeled datasets, which are computationally expensive to generate.
  2. Optimization Bottlenecks: Physics-Informed Neural Networks (PINNs) don't need labeled data but suffer from "stiff" optimization landscapes. Training a PINN for a single nonlinear PDE can take hours or even days, and they often fail to generalize to new parameters or initial conditions without costly retraining.

The authors tackle the specific challenge of meta-learning nonlinear PDEs—training a model that can "learn how to learn" a family of equations so that it can adapt to a new, unseen instance (e.g., a different viscosity or initial state) almost instantaneously.

Methods

Newton-PINet introduces a three-pronged architectural and algorithmic approach:

  1. Skip-Connected Tikhonov Architecture: The model uses a multilayer network where every hidden layer has a skip connection directly to the output. This creates a "richer basis space," effectively allowing the model to use features from different levels of abstraction. The final output weights are solved using Tikhonov regularization (a form of ridge regression), which provides a stable, closed-form solution for linear problems.
  2. Two-Stage Meta-Learning:
    • Stage 1 (Pre-training): The model uses gradient descent to learn the "task-agnostic" weights of the hidden layers and key hyperparameters.
  3. Stage 2 (Fast Inference): For a new task, the hidden layers are frozen. Only the output layer weights are updated using a gradient-free least-squares solve. This makes adaptation incredibly fast.
  4. Newton Linearization: This is the "secret sauce" for nonlinear problems. Traditional methods often use Picard iteration (lagging-of-coefficients), which converges slowly (linearly). Newton-PINet incorporates a Taylor-expansion-based Newton linearization, enabling near-quadratic convergence. This allows the model to solve complex nonlinear terms (like $u \frac{\partial u}{\partial x}$) in just a few iterations.
The Newton-PINet framework: Stage 1 meta-learns the representation, while Stage 2 performs fast, gradient-free adaptation to new tasks.

Datasets

The authors evaluated Newton-PINet on a diverse set of challenging nonlinear benchmarks:

  • Convection-type PDEs: Burgers’ equation, Generalized Korteweg-de Vries (KdV), and the chaotic Kuramoto-Sivashinsky (K-S) equation.
  • Forcing-type PDEs: Convection-Diffusion-Reaction (CDR), Klein-Gordon, and Hyperbolic/Logarithmic heat equations.
  • Higher Dimensions: 2D Lid-driven cavity (Navier-Stokes) and 2D/3D Parametric diffusion-reaction equations.
  • Irregular Geometries: Generalization tests on complex, non-rectangular domains.

Key Insights and Ideas

  • Newton > Picard: The shift from Picard to Newton linearization is transformative. As shown in the paper's derivations (Table 1), Newton linearization captures higher-order corrections that Picard misses, leading to much faster convergence during both training and inference.
  • Skip Connections as Basis Functions: Instead of just helping gradient flow (as in ResNets), skip connections here serve to stack nonlinear features, creating a wider output layer that makes the least-squares solve more expressive and numerically stable.
  • Hybrid Learning: While the model can be trained purely on physics (unsupervised), the authors found that a "hybrid" approach—using a small amount of labeled data in the outer meta-learning loop—significantly boosts stability for highly complex PDEs.
Table 1: Newton-linearized expressions of several nonlinear terms derived by our work. m denotes the exponent. k and k + 1 refer to the current and next nonlinear iterations, respectively.

Key Results

  • Speed: On a 1D nonlinear reaction-diffusion problem, Newton-PINet trained in 301 seconds (approx. 5 mins), compared to 4.5 hours for the PI-neural-solver baseline.
  • Accuracy: It achieved a test MSE of 1.72e-7, roughly three orders of magnitude lower than the baseline (2.91e-4).
  • Data Efficiency: The model reached these results using only 50 training tasks, whereas the baseline required 800.
  • Convergence: In Burgers' equation tests, Newton-PINet showed near-quadratic convergence (Empirical Order of Convergence $\approx 1.8-2.0$), while the Picard-based PINet remained stuck with linear convergence ($\approx 1.2$).
Figure 2: Burgers’ problem. (a) Meta-learning convergence of Newton-PINet vs. PINet under differ- ent nonlinear iterations (N) (lines and shaded areas: the median convergence path and interquartile ranges of 5 random-seed runs). (b) Boxplot of error distributions across all test tasks and predictions by Newton-PINet (N=6).

Limitations and Future Work

  • Matrix Scalability: The Tikhonov solve involves a matrix $\mathbf{A}$ that is dense. For very large 3D+time problems, the memory cost of this matrix could become a bottleneck. The authors suggest exploring low-rank strategies or iterative solvers for these cases.
  • Chaotic Systems: For highly chaotic systems like Kuramoto-Sivashinsky, the model still requires temporal domain decomposition (breaking the time horizon into blocks) to maintain accuracy. Achieving "single-shot" long-term prediction for chaos remains an open challenge.
  • Initial Guess Sensitivity: Like all Newton-based methods, the performance depends on having a reasonable initial guess, though the authors mitigate this by using the initial condition as a starting point.

Summary updated

Abstract

Scientific machine learning has opened new avenues for solving parameterized partial differential equations (PDEs), enabling models to learn a family of PDEs and generalize to unseen instances. In this context, data-driven operator learning methods typically require large training datasets, while physics-informed neural networks (PINNs) suffer from difficult optimization and limited generalization, especially for nonlinear PDEs. We propose Newton-PINet, a physics-informed network enhanced by Newton linearization, offering an effective meta-learning framework for nonlinear PDEs. Newton-PINet (i) employs a physics-informed multilayer network with skip connections, where the output-layer weights are solved by least squares; (ii) adopts a two-stage learning strategy that first leverages gradient-based training to learn robust representations from the available training tasks, and then performs gradient-free fine-tuning on the output layer for fast task-specific generalization; and (iii) incorporates a Newton linearization method to speed up the least-squares iteration for nonlinear PDE problems. On a challenging nonlinear reaction-diffusion benchmark, Newton-PINet achieves up to three orders of magnitude lower relative error than recent neural solvers, while using 16x fewer training tasks and over an order of magnitude less training time (about 5 minutes versus several hours). This work advances the meta-learning of PINNs toward data-efficient, fast, and generalizable physics solvers. The code and datasets are publicly available at: https://github.com/fycAIwj/Newton-PINet.
BibTeX
@misc{fan2026newton,
  author = {Yuchen Fan and Chang Wei and Pao-Hsiung Chiu and Chin Chun Ooi and Heyang Wang and Jian Cheng Wong},
  title = {{Newton-PINet: A fast physics-informed neural network with Newton linearization for meta-learning nonlinear PDEs}},
  year = {2026},
  eprint = {2611.20609},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.20609},
}

Loading related content...

Figure