Point-Calibrated Spectral Neural Operators

Xihang Yue, Linchao Zhu, Yi Yang

Abstract: Two typical neural models have been extensively studied for operator learning, learning in spatial space via attention mechanism or learning in spectral space via spectral analysis technique such as Fourier Transform. Spatial learning enables point-level flexibility but lacks global continuity constraint, while spectral learning enforces spectral continuity prior but lacks point-wise adaptivity. This work innovatively combines the continuity prior and the point-level flexibility, with the introduced Point-Calibrated Spectral Transform. It achieves this by calibrating the preset spectral eigenfunctions with the predicted point-wise frequency preference via neural gate mechanism. Beyond this, we introduce Point-Calibrated Spectral Neural Operators, which learn operator mappings by approximating functions with the point-level adaptive spectral basis, thereby not only preserving the benefits of spectral prior but also boasting the superior adaptability comparable to the attention mechanism. Comprehensive experiments demonstrate its consistent performance enhancement in extensive PDE solving scenarios.

Introduction: The Spectral-Physical Dichotomy

Solving Partial Differential Equations (PDEs) is fundamental to science and engineering, yet traditional numerical solvers often lack the speed required for real-time applications. Neural operators have emerged as a powerful data-driven alternative, but current architectures generally fall into two competing paradigms:

  1. Spectral-based methods (e.g., FNO): These utilize fixed basis functions, such as Fourier transforms, to capture global domain structures. While data-efficient and resolution-invariant, they are "rigid," struggling with fine-scale local variations because the same spectral filters are applied globally.
  2. Attention-based methods (e.g., Transolver): These rely on point-wise learning to capture complex local dynamics. Although highly flexible, they often lack the global smoothness priors of spectral methods, requiring vast amounts of data and frequently failing to generalize across different mesh resolutions.
Comparison of Spectral, Attention, and Holistic Physics Mixers.

This paper introduces the Holistic Physics Mixer (HPM) to bridge this gap, eliminating the trade-off by creating a framework where global continuity and local flexibility coexist in a single unified space.

Methodology: Holistic Spectral Space

The core innovation of this work is the transition from fixed spectral spaces to a Holistic Spectral Space. Instead of forcing a choice between spectral or physical domains, HPM modulates global spectral priors with local physical adaptivity.

Adaptive Eigenfunctions

Traditional spectral methods use a fixed basis $\Phi$. HPM introduces a coupling function $H(x, \Phi)$ that modulates these basis functions using local physical features $x$. The preferred implementation uses a point-wise neural gate mechanism: $H(x, \Phi) = \psi(\text{MLP}(x)) \odot \Phi$. This allows every spatial point to "decide" which spectral components to emphasize based on its local state.

Detailed architecture of the Holistic Physics Mixer and the concept of Holistic Spectral Eigenfunctions.

A Unified Framework

The authors mathematically demonstrate that both fixed spectral operators (like FNO) and linear attention mechanisms are actually "degraded" or special cases of the HPM framework. By integrating point-level physics status with domain-level spectral structures, HPM maintains a continuous global prior while "sharpening" its focus on boundaries or high-frequency turbulence.

To handle complex, irregular geometries such as airfoils or biological vessels, HPM utilizes Laplace-Beltrami Operator (LBO) eigenfunctions as its basis. This ensures the model is equally effective on structured grids and unstructured triangular meshes.

Experimental Results

The model was evaluated against state-of-the-art baselines (including FNO, GNOT, and Transolver) across an extensive suite of PDE problems, including Darcy Flow, Navier-Stokes, and blood flow simulations.

  • Superior Accuracy: HPM consistently outperformed all baselines. In Navier-Stokes simulations, HPM achieved a relative L2 error of 7.34e-2, significantly lower than Transolver's 9.00e-2.
  • Zero-Shot Generalization: HPM demonstrates remarkable resolution invariance. When trained on a $221 \times 51$ resolution and tested on a much coarser $45 \times 11$ grid for the Airfoil task, HPM maintained an error of 3.85e-2, while Transolver's error spiked to 1.22e-1.
  • Computational Efficiency: Despite its superior performance, HPM is approximately 1.7x faster than Transolver and utilizes nearly 4x fewer parameters, making it highly suitable for industrial-scale applications.

Conclusion

The Point-Calibrated Spectral Neural Operator represents a significant step forward in operator learning. By unifying the global constraints of spectral analysis with the local flexibility of attention mechanisms, it provides a robust, efficient, and highly generalizable solver for complex physical systems. Future work will explore more sophisticated task-specific coupling mechanisms and the application of this holistic transform to broader fields like time-series prediction and computer vision.

Summary updated

Abstract

Two typical neural models have been extensively studied for operator learning, learning in spatial space via attention mechanism or learning in spectral space via spectral analysis technique such as Fourier Transform. Spatial learning enables point-level flexibility but lacks global continuity constraint, while spectral learning enforces spectral continuity prior but lacks point-wise adaptivity. This work innovatively combines the continuity prior and the point-level flexibility, with the introduced Point-Calibrated Spectral Transform. It achieves this by calibrating the preset spectral eigenfunctions with the predicted point-wise frequency preference via neural gate mechanism. Beyond this, we introduce Point-Calibrated Spectral Neural Operators, which learn operator mappings by approximating functions with the point-level adaptive spectral basis, thereby not only preserving the benefits of spectral prior but also boasting the superior adaptability comparable to the attention mechanism. Comprehensive experiments demonstrate its consistent performance enhancement in extensive PDE solving scenarios.
BibTeX
@misc{yue2024point,
  author = {Xihang Yue and Linchao Zhu and Yi Yang},
  title = {{Point-Calibrated Spectral Neural Operators}},
  year = {2024},
  eprint = {2410.11382},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.11382},
}
Figure