Abstract
Physics-informed machine learning typically integrates physical priors into the learning process by minimizing a loss function that includes both a data-driven term and a partial differential equation (PDE) regularization. Building on the formulation of the problem as a kernel regression task, we use Fourier methods to approximate the associated kernel, and propose a tractable estimator that minimizes the physics-informed risk function. We refer to this approach as physics-informed kernel learning (PIKL). This framework provides theoretical guarantees, enabling the quantification of the physical prior's impact on convergence speed. We demonstrate the numerical performance of the PIKL estimator through simulations, both in the context of hybrid modeling and in solving PDEs. In particular, we show that PIKL can outperform physics-informed neural networks in terms of both accuracy and computation time. Additionally, we identify cases where PIKL surpasses traditional PDE solvers, particularly in scenarios with noisy boundary conditions.
Introduction
Physics-Informed Machine Learning (PIML) has emerged as a vital field for blending empirical data with fundamental physical laws, typically expressed as partial differential equations (PDEs). While Physics-Informed Neural Networks (PINNs) have become the standard tool for these tasks, they present significant practical challenges. Training PINNs often requires complex optimization strategies to avoid local minima, and the "black-box" nature of neural networks makes it difficult to provide rigorous theoretical guarantees on convergence or to quantify the benefit of the physical prior.
Physics-Informed Kernel Learning (PIKL) addresses these issues by reframing the problem as a kernel regression task. By leveraging the stability and interpretability of kernel methods, PIKL offers a mathematically grounded alternative that is both faster to train and more accurate than traditional PINN architectures.
Methodology
The core innovation of PIKL lies in its treatment of the physics-informed kernel. While the exact kernel required for a specific PDE is often analytically intractable, PIKL utilizes Fourier methods for approximation.
Fourier Approximation and Linear Solvers
Instead of solving a weak PDE to find the kernel, PIKL expands the kernel into a truncated Fourier series. Because differential operators act linearly on Fourier modes, the optimization problem transforms from a complex non-convex landscape into a system of linear equations. This allows for a closed-form solution, eliminating the need for backpropagation or hyperparameter tuning of learning rates.
Theoretical Guarantees
PIKL introduces the concept of "effective dimension" to theoretically quantify how a physical prior accelerates learning. The authors prove that incorporating physics can shift the convergence rate from the standard Sobolev minimax rate to a much faster parametric rate. This provides a clear mathematical justification for the efficiency of physics-informed models.
Experimental Evaluation
The performance of PIKL was evaluated across several synthetic benchmarks:
- Harmonic Oscillator (1D): A baseline for modeling accuracy.
- Heat Equation (2D): Tested on various domain shapes, including squares and disks.
- Convection Equation: A challenging case for PINNs due to high-frequency oscillations.
- Wave Equation (2D): A complex benchmark for comparing PIKL against state-of-the-art PINN architectures.
Results and Discussion
PIKL demonstrates significant advantages over both neural network approaches and traditional numerical solvers.
Accuracy and Efficiency
In solving the Wave Equation, PIKL achieved a relative $L^2$ error of $10^{-8}$, whereas vanilla PINNs reached only $10^{-1}$ and even highly optimized PINNs plateaued at $10^{-3}$. Furthermore, PIKL is remarkably efficient; for a 1D wave equation, it trained in just 6 seconds on an Nvidia L4 GPU—a 70x speedup compared to the 7 minutes required for a vanilla PINN.
Robustness to Noise
A key strength of PIKL is its statistical nature. Unlike traditional PDE solvers like Crank-Nicolson, which are highly sensitive to noise in boundary conditions, PIKL naturally filters noise. In tests with Gaussian noise on boundary conditions, PIKL outperformed traditional solvers, achieving an error of $1.87 \times 10^{-2}$ compared to $2.01 \times 10^{-2}$ for the best classical method.
Conclusion
PIKL provides a robust, efficient, and theoretically sound framework for physics-informed learning. By moving from the "art" of neural network optimization to the clarity of kernel methods and Fourier analysis, it achieves superior accuracy and speed. While currently focused on linear PDEs with constant coefficients, future work aims to extend this approach to non-linear systems and more complex engineering geometries.
Summary updated