Abstract
Neural Operators, such as the Fourier Neural Operator (FNO) and Deep Operator Network (DeepONet), have revolutionized the data-driven solution of partial differential equations (PDEs). However, these models often function as "black boxes" with systematic errors stemming from grid-based sampling. This paper introduces Spectral Neural Operators (SNO), which map functions directly in the space of their spectral coefficients. By leveraging Chebyshev or Fourier bases, SNO eliminates aliasing bias, ensures mathematical transparency, and achieves state-of-the-art accuracy on smooth PDE benchmarks.
Introduction: The Limitations of Grid-Based Operators
While FNO and DeepONet are powerful tools for learning mappings between infinite-dimensional function spaces, they suffer from two primary drawbacks: opaque outputs and aliasing bias. Most neural operators represent their output as a standard neural network, making it difficult to extract rigorous derivatives or global bounds—tasks that are trivial in classical numerical methods like Finite Element Methods (FEM).
More critically, grid-based methods like FNO are prone to aliasing. When nonlinear activation functions (e.g., ReLU) are applied to a function sampled on a grid, they generate high-frequency components. If the grid resolution is insufficient to resolve these frequencies, they "alias" back into lower frequencies, creating a persistent error floor that additional data cannot easily overcome.
The Spectral Neural Operator (SNO) Framework
The proposed SNO architecture shifts the learning process from the spatial domain to the coefficient domain. Instead of sampling functions on a grid, SNO treats functions as vectors of coefficients in a truncated series—using Chebyshev polynomials for non-periodic domains or Fourier series for periodic ones.
Architecture and Operations
SNO utilizes the duality between grid values and spectral coefficients via Fast Fourier Transforms (FFT) and Discrete Cosine Transforms (DCT). The model architecture consists of three main components:
- Feature Expansion ($N_1$): Increases the feature count of the input coefficients.
- Integral Operation ($N_2$): Performs a low-rank operation directly in the coefficient space.
- Projection ($N_3$): Maps the features back to the desired output dimension.
Because SNO operates on coefficients, fundamental operations like differentiation, integration, and shifts are performed with machine precision using standard recurrence relations, bypassing the approximation errors inherent in finite difference methods.
Key Insights: The Aliasing Theorem and Super-Resolution
A central contribution of this work is the formalization of the "Aliasing Theorem." The authors provide a formal proof (Theorem 1) showing that applying a ReLU activation to a function on a grid can result in a relative aliasing error of approximately 31%. This theoretical bound explains why grid-based operators often hit an accuracy plateau regardless of the training grid density.
Furthermore, the paper challenges the common claim of "zero-shot super-resolution." The authors argue that many existing tests merely demonstrate better interpolation on finer grids. By training on low-frequency harmonics and testing on high-frequency ones, they show that current operators (including SNO) cannot truly "discover" high-frequency physics absent from the training set, highlighting a fundamental limitation in the field.
Experimental Results
SNO was evaluated against FNO and DeepONet across 16 benchmarks, including linear operators, nonlinear ODEs, and complex PDEs like the Burgers’, KdV, and Nonlinear Schrödinger equations.
- Superior Accuracy: On smooth benchmarks (e.g., KdV, Breather), SNO achieved test errors 1–2 orders of magnitude lower than its competitors. For 1D differentiation, SNO's error was $1.5 \times 10^{-4}$, significantly outperforming FNO ($5.0 \times 10^{-4}$) and DeepONet ($2.5 \times 10^{-2}$).
- Consistency: SNO maintains high accuracy across different grid resolutions, whereas FNO's performance is constrained by the aforementioned aliasing bias.
- The Gibbs Phenomenon: SNO faces challenges with non-smooth solutions, such as shocks in the viscous Burgers’ equation at low viscosity ($\nu = 0.01$). In these cases, the spectral representation leads to oscillations, a well-known limitation of global spectral methods.
Conclusion and Future Directions
Spectral Neural Operators bridge the gap between deep learning and classical numerical analysis. By representing functions in coefficient space, SNO provides a rigorous, interpretable, and highly accurate alternative to grid-based operators for smooth problems. Future work aims to address discontinuities through adaptive bases and to integrate SNO with classical spectral libraries like Chebfun to create a fully realized "Neural Chebfun."
Summary updated