Fourier Continuation for Exact Derivative Computation in Physics-Informed Neural Operators

Haydn Maust, Zongyi Li, Yixuan Wang, Daniel V. Leibovici, Oscar P. Bruno, Thomas Y. Hou, Anima AnandkumararXiv 2022 · DOI · arXiv

Introduction

Physics-Informed Neural Operators (PINOs) represent a significant advancement in scientific machine learning, combining the data-driven power of neural operators with the physical constraints of partial differential equations (PDEs). A critical challenge in training PINOs is the computation of the "physics loss," which requires calculating derivatives of the model's output.

Traditionally, researchers have faced a difficult choice between three methods for derivative calculation:

  1. Finite Differences: Simple to implement but requires high resolution for accuracy and struggles with sharp gradients.
  2. Automatic Differentiation (AD): Highly accurate but extremely memory-intensive, often creating a bottleneck for large 3D simulations.
  3. Spectral Differentiation: Utilizing Fast Fourier Transforms (FFTs) for speed and memory efficiency, but limited by the assumption of periodicity.

When spectral methods are applied to non-periodic domains, they suffer from the Gibbs phenomenon—spurious oscillations at the boundaries that degrade the precision of the solution. This paper introduces FC-PINO, a novel architecture that integrates Fourier Continuation (FC) to enable high-precision spectral differentiation for non-periodic problems.

Methods: The FC-PINO Architecture

The core innovation of FC-PINO is the embedding of a Fourier Continuation layer within the neural operator framework. Instead of forcing a non-periodic signal into a periodic box, which creates discontinuities at the edges, Fourier Continuation uses polynomial interpolation to create a smooth, periodic extension of the signal onto an enlarged domain.

The FC-PINO architecture integrates Fourier layers with a specific internal structure to process extended periodic domains.

The Workflow

  1. Extension: The model applies FC to extend the input signal into a larger, periodic domain.
  2. Processing: A Fourier Neural Operator (FNO) runs on this extended domain.
  3. Restriction: The output is restricted back to the original physical domain.
  4. Derivative Calculation: Because the internal representation is periodic on the extended domain, the authors use the chain rule combined with spectral differentiation to obtain exact, high-precision derivatives without boundary artifacts.

Flavors of Fourier Continuation

The paper explores two specific extension methods:

  • FC-Legendre: Employs a global fit with Legendre polynomials, ideal for very smooth, noise-free signals.
  • FC-Gram: Uses localized Gram polynomial bases and blending, providing greater robustness to noise and non-smooth data.
Comparison of 1D extensions: FC-Legendre and FC-Gram create smooth, periodic representations of non-periodic functions.

Key Results and Insights

The authors evaluated FC-PINO on three challenging non-periodic PDE benchmarks: the 1D inviscid Burgers’ equation, the 2D viscous Burgers’ equation, and the 3D incompressible Navier-Stokes equations (specifically the lid-driven cavity flow).

  • Precision Breakthrough: Standard PINOs often stall at residuals between $10^{-3}$ and $10^{-5}$ due to boundary errors. FC-PINO successfully reduced PDE residuals on the 1D Burgers' equation to $10^{-12}$, outperforming baselines by several orders of magnitude.
  • Scalability: By avoiding memory-heavy Automatic Differentiation for the physics loss, FC-PINO remains computationally feasible for large-scale 3D problems where AD would typically exceed GPU memory limits.
  • Robustness in 3D: In the 3D Navier-Stokes test, standard PINO and simple padding-based versions failed to satisfy the governing equations correctly. FC-PINO converged to the correct solution with minimal residuals, demonstrating that mathematically sound boundary treatment accelerates the learning process.

Conclusion

FC-PINO addresses a fundamental limitation in physics-informed learning by enabling the use of spectral methods on non-periodic domains. By eliminating the Gibbs phenomenon through Fourier Continuation, the model achieves the high precision necessary for rigorous scientific research, such as investigating the finite-time blowup of Navier-Stokes equations. Future work will focus on refining hyperparameter selection for extension lengths and polynomial degrees to further enhance robustness across varying grid resolutions.

Summary updated

Abstract

The physics-informed neural operator (PINO) is a machine learning paradigm that has demonstrated promising results for learning solutions to partial differential equations (PDEs). It leverages the Fourier Neural Operator to learn solution operators in function spaces and leverages physics losses during training to penalize deviations from known physics laws. Spectral differentiation provides an efficient way to compute derivatives for the physics losses, but it inherently assumes periodicity. When applied to non-periodic functions, this assumption can lead to significant errors, including Gibbs phenomena near domain boundaries which degrade the accuracy of both function representations and derivative computations. To overcome this limitation, we introduce the FC-PINO (Fourier-Continuation-based Physics-Informed Neural Operator) architecture which extends the accuracy and efficiency of PINO and spectral differentiation to non-periodic and non-smooth PDEs. In FC-PINO, we propose integrating Fourier continuation into the PINO framework, and test two different continuation approaches: FC-Legendre and FC-Gram. By transforming non-periodic signals into periodic functions on extended domains in a well-conditioned manner, Fourier continuation enables fast and accurate derivative computations. This approach avoids the discretization sensitivity of finite differences and the memory overhead of automatic differentiation. We demonstrate that standard PINO fails (without padding) or struggles (even with padding) to solve non-periodic and non-smooth PDEs with high precision, across challenging benchmarks. In contrast, the proposed FC-PINO provides accurate, robust, and scalable solutions, substantially outperforming PINO alternatives, and demonstrating that Fourier continuation is critical for extending PINO to a wider range of PDE problems when high-precision solutions are needed.
BibTeX
@misc{maust2022fourier,
  author = {Haydn Maust and Zongyi Li and Yixuan Wang and Daniel V. Leibovici and Oscar P. Bruno and Thomas Y. Hou and Anima Anandkumar},
  title = {{Fourier Continuation for Exact Derivative Computation in Physics-Informed Neural Operators}},
  year = {2022},
  eprint = {2211.15960},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2211.15960},
}
Figure