Abstract
Physics-Informed Neural Networks (PINNs) have emerged as a powerful tool for solving partial differential equations (PDEs) without the need for traditional meshes. However, their performance is often hindered by inefficient sampling and convergence issues in complex domains. This paper introduces an iterative refinement strategy that leverages Optimal Transport (OT) to intelligently redistribute collocation points. By focusing training efforts on regions with high PDE residuals, the proposed method significantly enhances sample efficiency and accuracy. We demonstrate the effectiveness of this approach on the Fokker-Planck-Kolmogorov Equation (FPKE) for nonlinear stochastic systems, achieving high precision with a fraction of the training data required by standard PINNs.
Introduction
Solving partial differential equations (PDEs) is fundamental to science and engineering, yet traditional numerical methods like finite elements often succumb to the "curse of dimensionality." Physics-Informed Neural Networks (PINNs) offer a promising alternative by using neural networks to approximate PDE solutions directly. Despite their flexibility, PINNs face challenges such as spectral bias—a tendency to learn low-frequency components while missing sharp gradients—and sampling inefficiencies when using uniform or random grids.
This study focuses on the Fokker-Planck-Kolmogorov Equation (FPKE), which governs the evolution of probability distributions in stochastic systems. The FPKE is particularly challenging because its solution, the probability density $\rho(x)$, must remain non-negative and integrate to unity. Standard PINNs often struggle to satisfy these constraints, leading to unstable training and inaccurate results.
Methodology
Potential Function Transformation
To address the non-negativity constraint of the PDF, the authors employ a potential function transformation. Instead of approximating $\rho(x)$ directly, the network learns a potential function $\eta(x)$, where: $$\rho(x) = N_0 \exp(-\eta(x))$$ This formulation ensures that the probability density is strictly positive by construction. The normalization constant $N_0$ is determined through post-processing, transforming a constrained optimization problem into a more stable regression task.
Optimal Transport Based Refinement
The primary contribution of this work is an iterative "active learning" framework for point selection. The process begins with training a PINN on a sparse initial set of points. The refinement loop then proceeds as follows:
- Error Evaluation: The PDE residual (the "equation error") is calculated across the domain to identify regions of poor fit.
- OT Resampling: Optimal Transport theory is used to solve a linear programming problem that maps a uniform distribution of points to a target distribution proportional to the error magnitude. This effectively "pushes" collocation points from low-error regions to high-error regions.
- Iterative Training: These newly redistributed points are added to the training set, and the model is retrained.
Experimental Results
Van der Pol - Rayleigh Oscillator
The method was first validated on the Van der Pol - Rayleigh oscillator, a 2D system with a known analytical solution. The OT-based refinement demonstrated exceptional sample efficiency. As shown in Figure 1, the model achieved a solution error ($\epsilon_\rho$) of approximately $1.93 \times 10^{-5}$. Remarkably, the OT-PINN reached this level of accuracy using only 1,225 training points, compared to the 6,241 points required by a standard baseline PINN to achieve similar results—a reduction of over 80% in required data.
Classic Van der Pol Oscillator
For the more complex classic Van der Pol oscillator, which lacks an analytical solution, the authors relied on the PDE residual ($\epsilon_{pde}$) to measure performance. Initial training resulted in significant residuals, but after 10 iterations of OT-based refinement, the error dropped from $O(10^5)$ to $O(10^{-3})$. Figure 2 illustrates the steady-state solution and the remaining equation error, highlighting the model's ability to capture the intricate geometry of the attractor.
Conclusion
The integration of Optimal Transport into the PINN training pipeline provides a principled and effective method for adaptive sampling. By dynamically redistributing collocation points toward high-error regions, the OT-refinement strategy overcomes common convergence issues and dramatically improves sample efficiency. Future research will focus on scaling this approach to higher-dimensional stochastic systems and investigating the sensitivity of the refinement process to initial network weights and hyperparameters.
Summary updated