Finite basis physics-informed neural networks (FBPINNs): a scalable domain decomposition approach for solving differential equations

Ben Moseley, Andrew Markham, Tarje Nissen-Meyer

Introduction

Physics-Informed Neural Networks (PINNs) have emerged as a transformative tool for solving partial differential equations (PDEs). By embedding physical laws directly into the loss function of a neural network, PINNs provide a mesh-free framework capable of handling both forward and inverse problems. However, despite their versatility, standard PINNs face significant hurdles when applied to large-scale domains or problems involving multi-scale phenomena. As the complexity of the solution increases, PINNs often struggle with convergence and accuracy.

To address these limitations, the research introduces Finite Basis Physics-Informed Neural Networks (FBPINNs). This framework presents a scalable domain decomposition strategy inspired by classical finite element methods. By decomposing the global problem into localized subproblems handled by individual neural networks, FBPINNs overcome the fundamental bottlenecks of global PINN architectures.

The Scaling Problem in Standard PINNs

The failure of standard PINNs in large-scale applications is primarily attributed to two factors:

  1. Optimization Complexity: Representing a complex global solution with a single, large neural network creates a highly non-convex and "stiff" optimization landscape. The optimizer often gets trapped in poor local minima, failing to capture the intricate details of the physical system.
  2. Spectral Bias: Neural networks exhibit a "spectral bias," where they preferentially learn low-frequency components of a signal before high-frequency ones. In large domains, high-frequency physical features are compressed relative to the global input range (typically normalized to $[-1, 1]$), making them extremely difficult for a global network to resolve.

The FBPINN Framework

FBPINNs redefine the solution ansatz by combining domain decomposition with localized basis functions.

Domain Decomposition and Windowing

The global domain is partitioned into a set of overlapping subdomains. For each subdomain, a dedicated "expert" neural network is assigned. To ensure a smooth global solution, each local network is multiplied by a differentiable window function (typically based on sigmoids). The global solution is then the sum of these windowed local solutions. This construction ensures $C^\infty$ continuity across the entire domain without requiring the complex interface penalty terms or "gluing" constraints common in other decomposition methods like XPINNs.

Local Normalization: Overcoming Spectral Bias

The "secret sauce" of FBPINNs is local normalization. Each subdomain network receives inputs that are normalized specifically to its local coordinate system. This effectively "stretches" high-frequency physical features back into a range that the neural network can easily process. By localizing the input space, FBPINNs transform a globally high-frequency problem into a series of locally low-frequency problems, directly bypassing the spectral bias limitation.

Training Strategies

FBPINNs allow for flexible training schedules. For problems with well-defined boundary or initial conditions, the authors propose "learning outwards." Subdomains near the boundaries are trained first; once they converge, their parameters can be frozen or regularized while the optimization proceeds to adjacent subdomains. This prevents the optimizer from exploring physically inconsistent regions of the solution space far from the constraints.

Experimental Evaluation

The FBPINN framework was validated against several challenging benchmarks:

  • High-Frequency Sinusoids: In 1D and 2D tests with high-frequency oscillations ($\omega=15$), standard PINNs failed to converge, while FBPINNs achieved high accuracy with significantly fewer training iterations.
  • Viscous Burgers Equation: FBPINNs successfully captured the sharp gradients and shock-like behavior characteristic of the Burgers equation, demonstrating robustness in the presence of discontinuities.
  • Acoustic Wave Equation: In a (2+1)D simulation of a point source in a non-uniform medium, FBPINNs outperformed global PINNs in both stability and efficiency. While standard PINNs often collapsed to a trivial zero solution, FBPINNs robustly captured the propagating wavefield.

Key Insights and Advantages

  • Computational Efficiency: FBPINNs are inherently parallel. Each subdomain network can be trained independently on separate hardware units, with minimal communication overhead. Furthermore, the authors found that FBPINNs require significantly fewer total floating-point operations (FLOPs) to reach target accuracy compared to global networks.
  • Robustness: The localized nature of the optimization makes the system less sensitive to initialization and hyperparameter choices.
  • Continuity by Design: The overlapping window function approach provides a naturally smooth global field, simplifying the implementation compared to other domain decomposition methods.

Future Directions

While FBPINNs show great promise, several areas for future research remain. The current implementation uses a fixed grid of subdomains; moving toward adaptive "h-p refinement"—where subdomains are automatically added or network sizes increased in regions of high error—could further enhance efficiency. Additionally, extending the framework to irregular subdomain geometries will be crucial for solving problems in complex 3D manifolds.

Summary updated

Abstract

Recently, physics-informed neural networks (PINNs) have offered a powerful new paradigm for solving problems relating to differential equations. Compared to classical numerical methods, PINNs have several advantages, for example their ability to provide mesh-free solutions of differential equations and their ability to carry out forward and inverse modelling within the same optimisation problem. Whilst promising, a key limitation to date is that PINNs have struggled to accurately and efficiently solve problems with large domains and/or multi-scale solutions, which is crucial for their real-world application. Multiple significant and related factors contribute to this issue, including the increasing complexity of the underlying PINN optimisation problem as the problem size grows and the spectral bias of neural networks. In this work, we propose a new, scalable approach for solving large problems relating to differential equations called finite basis physics-informed neural networks (FBPINNs). FBPINNs are inspired by classical finite element methods, where the solution of the differential equation is expressed as the sum of a finite set of basis functions with compact support. In FBPINNs, neural networks are used to learn these basis functions, which are defined over small, overlapping subdomains. FBINNs are designed to address the spectral bias of neural networks by using separate input normalisation over each subdomain and reduce the complexity of the underlying optimisation problem by using many smaller neural networks in a parallel divide-and-conquer approach. Our numerical experiments show that FBPINNs are effective in solving both small and larger, multi-scale problems, outperforming standard PINNs in both accuracy and computational resources required, potentially paving the way to the application of PINNs on large, real-world problems.
BibTeX
@inproceedings{moseley2023finite,
  author = {Ben Moseley and Andrew Markham and Tarje Nissen-Meyer},
  title = {{Finite basis physics-informed neural networks (FBPINNs): a scalable domain decomposition approach for solving differential equations}},
  booktitle = {Adv. Comput. Math},
  year = {2023},
  url = {https://openreview.net/forum?id=66rxkiQ3AN},
}
Figure