Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equations

Ben Moseley, Andrew Markham, Tarje Nissen-Meyer

Abstract

Recently, physics-informed neural networks (PINNs) have offered a powerful new paradigm for solving problems relating to differential equations. Compared to classical numerical methods PINNs have several advantages, for example their ability to provide mesh-free solutions of differential equations and their ability to carry out forward and inverse modelling within the same optimisation problem. Whilst promising, a key limitation to date is that PINNs have struggled to accurately and efficiently solve problems with large domains and/or multi-scale solutions, which is crucial for their real-world application. Multiple significant and related factors contribute to this issue, including the increasing complexity of the underlying PINN optimisation problem as the problem size grows and the spectral bias of neural networks. In this work we propose a new, scalable approach for solving large problems relating to differential equations called Finite Basis PINNs (FBPINNs). FBPINNs are inspired by classical finite element methods, where the solution of the differential equation is expressed as the sum of a finite set of basis functions with compact support. In FBPINNs neural networks are used to learn these basis functions, which are defined over small, overlapping subdomains. FBINNs are designed to address the spectral bias of neural networks by using separate input normalisation over each subdomain, and reduce the complexity of the underlying optimisation problem by using many smaller neural networks in a parallel divide-and-conquer approach. Our numerical experiments show that FBPINNs are effective in solving both small and larger, multi-scale problems, outperforming standard PINNs in both accuracy and computational resources required, potentially paving the way to the application of PINNs on large, real-world problems.

Introduction: The Scalability Challenge

Physics-Informed Neural Networks (PINNs) have emerged as a revolutionary tool for solving differential equations without needing a traditional mesh. However, they face a "Goldilocks" problem: they work beautifully on small, simple domains but often fall apart when applied to large-scale, real-world problems.

The authors identify two primary culprits for this failure:

  1. Optimization Complexity: As the domain grows, the neural network must become larger and more complex. Training one giant network to satisfy a PDE across a massive space is a significant challenge for gradient descent.
  2. Spectral Bias: Neural networks are naturally biased toward learning low-frequency patterns first. In a large domain, even a "low-frequency" physical feature can appear as a high-frequency signal relative to the global input normalization, causing the network to struggle or fail to converge.
Standard PINNs struggle to capture high-frequency solutions (omega=15) even as network size increases, illustrating the spectral bias and optimization challenges that FBPINNs aim to solve.

Methods: Finite Basis PINNs (FBPINNs)

The proposed Finite Basis PINNs (FBPINNs) draw inspiration from classical Finite Element Methods (FEM) to overcome these limitations through domain decomposition.

The Core Architecture

Instead of one network for the whole domain, the space is divided into many small, overlapping subdomains. Each subdomain is assigned its own small neural network, confined to its specific region using a smooth, differentiable window function. The global solution is defined as the sum of these local networks. Because the subdomains overlap and the window functions are smooth, the global solution is naturally continuous without needing extra "interface loss" terms.

Local Normalization

A critical component of FBPINNs is subdomain normalization. Each local network sees inputs normalized specifically to its subdomain. This effectively transforms global high-frequency features into local low-frequency ones, directly bypassing the spectral bias issue.

Flexible Training Schedules

FBPINNs allow for "learning outwards." Training can start by optimizing networks near boundary conditions and slowly "activating" neighboring subdomains, mimicking time-marching schemes in classical physics.

Key Results

FBPINNs were tested against standard PINNs on several synthetic benchmarks, including sinusoidal problems, multi-scale 1D problems, the viscous Burgers equation, and the (2+1)D wave equation.

  • High-Frequency Mastery: In a 1D test with $\omega=15$, standard PINNs failed to capture the solution accurately even with massive networks. FBPINNs converged to a high-accuracy solution in a fraction of the training steps.
  • Computational Efficiency: FBPINNs required multiple orders of magnitude fewer FLOPS (Floating Point Operations) for forward inference during training compared to standard PINNs. This is because they use many tiny networks instead of one deep, wide network.
  • Robustness on the Wave Equation: On the complex (2+1)D wave equation, standard PINNs often got stuck in local minima (predicting a zero solution). FBPINNs, using a "learning outwards" schedule, robustly solved the problem and matched finite-difference ground truths.

Discussion and Future Work

While FBPINNs are more data-efficient (fewer FLOPS), the current single-threaded implementation is 2–10x slower in real time than standard PINNs. The authors note that a multi-threaded or multi-GPU implementation is needed to realize the true speed potential of this parallel architecture.

Furthermore, while the current work focuses on simple geometries, extending FBPINNs to complex, irregular shapes and higher-dimensional problems is a logical next step. Choosing hyperparameters like the number of subdomains and overlap remains a task requiring domain expertise, similar to mesh selection in traditional FEM.

Summary updated

Abstract

Recently, physics-informed neural networks (PINNs) have offered a powerful new paradigm for solving problems relating to differential equations. Compared to classical numerical methods PINNs have several advantages, for example their ability to provide mesh-free solutions of differential equations and their ability to carry out forward and inverse modelling within the same optimisation problem. Whilst promising, a key limitation to date is that PINNs have struggled to accurately and efficiently solve problems with large domains and/or multi-scale solutions, which is crucial for their real-world application. Multiple significant and related factors contribute to this issue, including the increasing complexity of the underlying PINN optimisation problem as the problem size grows and the spectral bias of neural networks. In this work we propose a new, scalable approach for solving large problems relating to differential equations called Finite Basis PINNs (FBPINNs). FBPINNs are inspired by classical finite element methods, where the solution of the differential equation is expressed as the sum of a finite set of basis functions with compact support. In FBPINNs neural networks are used to learn these basis functions, which are defined over small, overlapping subdomains. FBINNs are designed to address the spectral bias of neural networks by using separate input normalisation over each subdomain, and reduce the complexity of the underlying optimisation problem by using many smaller neural networks in a parallel divide-and-conquer approach. Our numerical experiments show that FBPINNs are effective in solving both small and larger, multi-scale problems, outperforming standard PINNs in both accuracy and computational resources required, potentially paving the way to the application of PINNs on large, real-world problems.
BibTeX
@inproceedings{moseley2023finite,
  author = {Ben Moseley and Andrew Markham and Tarje Nissen-Meyer},
  title = {{Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equations}},
  booktitle = {Adv. Comput. Math},
  year = {2023},
  url = {https://openreview.net/forum?id=66rxkiQ3AN},
}

Loading related content...

Figure