Introduction
Physics-Informed Neural Operators (PINOs) represent a significant advancement in scientific machine learning, combining the data-driven power of neural operators with the physical constraints of partial differential equations (PDEs). A critical challenge in training PINOs is the computation of the "physics loss," which requires calculating derivatives of the model's output.
Traditionally, researchers have faced a difficult choice between three methods for derivative calculation:
- Finite Differences: Simple to implement but requires high resolution for accuracy and struggles with sharp gradients.
- Automatic Differentiation (AD): Highly accurate but extremely memory-intensive, often creating a bottleneck for large 3D simulations.
- Spectral Differentiation: Utilizing Fast Fourier Transforms (FFTs) for speed and memory efficiency, but limited by the assumption of periodicity.
When spectral methods are applied to non-periodic domains, they suffer from the Gibbs phenomenon—spurious oscillations at the boundaries that degrade the precision of the solution. This paper introduces FC-PINO, a novel architecture that integrates Fourier Continuation (FC) to enable high-precision spectral differentiation for non-periodic problems.
Methods: The FC-PINO Architecture
The core innovation of FC-PINO is the embedding of a Fourier Continuation layer within the neural operator framework. Instead of forcing a non-periodic signal into a periodic box, which creates discontinuities at the edges, Fourier Continuation uses polynomial interpolation to create a smooth, periodic extension of the signal onto an enlarged domain.
The Workflow
- Extension: The model applies FC to extend the input signal into a larger, periodic domain.
- Processing: A Fourier Neural Operator (FNO) runs on this extended domain.
- Restriction: The output is restricted back to the original physical domain.
- Derivative Calculation: Because the internal representation is periodic on the extended domain, the authors use the chain rule combined with spectral differentiation to obtain exact, high-precision derivatives without boundary artifacts.
Flavors of Fourier Continuation
The paper explores two specific extension methods:
- FC-Legendre: Employs a global fit with Legendre polynomials, ideal for very smooth, noise-free signals.
- FC-Gram: Uses localized Gram polynomial bases and blending, providing greater robustness to noise and non-smooth data.
Key Results and Insights
The authors evaluated FC-PINO on three challenging non-periodic PDE benchmarks: the 1D inviscid Burgers’ equation, the 2D viscous Burgers’ equation, and the 3D incompressible Navier-Stokes equations (specifically the lid-driven cavity flow).
- Precision Breakthrough: Standard PINOs often stall at residuals between $10^{-3}$ and $10^{-5}$ due to boundary errors. FC-PINO successfully reduced PDE residuals on the 1D Burgers' equation to $10^{-12}$, outperforming baselines by several orders of magnitude.
- Scalability: By avoiding memory-heavy Automatic Differentiation for the physics loss, FC-PINO remains computationally feasible for large-scale 3D problems where AD would typically exceed GPU memory limits.
- Robustness in 3D: In the 3D Navier-Stokes test, standard PINO and simple padding-based versions failed to satisfy the governing equations correctly. FC-PINO converged to the correct solution with minimal residuals, demonstrating that mathematically sound boundary treatment accelerates the learning process.
Conclusion
FC-PINO addresses a fundamental limitation in physics-informed learning by enabling the use of spectral methods on non-periodic domains. By eliminating the Gibbs phenomenon through Fourier Continuation, the model achieves the high precision necessary for rigorous scientific research, such as investigating the finite-time blowup of Navier-Stokes equations. Future work will focus on refining hyperparameter selection for extension lengths and polynomial degrees to further enhance robustness across varying grid resolutions.
Summary updated