Abstract: Recent advances in operator learning have produced two distinct approaches for solving partial differential equations (PDEs): attention-based methods offering point-level adaptability but lacking spectral constraints, and spectral-based methods providing domain-level continuity priors but limited in local flexibility. This dichotomy has hindered the development of PDE solvers with both strong flexibility and generalization capability. This work introduces Holistic Physics Mixer (HPM), a simple framework that bridges this gap by integrating spectral and physical information in a unified space. HPM unifies both approaches as special cases while enabling more powerful spectral-physical interactions beyond either method alone. This enables HPM to inherit both the strong generalization of spectral methods and the flexibility of attention mechanisms while avoiding their respective limitations. Through extensive experiments across diverse PDE problems, we demonstrate that HPM consistently outperforms state-of-the-art methods in both accuracy and computational efficiency, while maintaining strong generalization capabilities with limited training data and excellent zero-shot performance on unseen resolutions.
Introduction
Solving partial differential equations (PDEs) is fundamental to science and engineering, yet traditional numerical solvers often face computational bottlenecks in real-time applications. Neural operators have emerged as a promising data-driven alternative, but the field has largely remained split into two competing paradigms:
- Spectral-based methods (e.g., FNO): These utilize fixed spectral bases to enforce global continuity. While data-efficient, they lack "point-level flexibility," struggling to adapt to local features or sharp variations.
- Attention-based methods (e.g., Transformers): These learn point-wise interactions, offering high flexibility. However, they often lack physical priors, leading to overfitting on small datasets and poor generalization across different spatial resolutions.
Methodology: The Holistic Physics Mixer
The core innovation of this work is the Holistic Physics Mixer (HPM), which unifies spectral information and physical states within a single "holistic spectral space." Instead of relying on a static spectral basis, HPM employs a coupling function $H(x, \Phi)$ that dynamically integrates local physical features ($x$) with a spectral basis ($\Phi$, such as Laplace-Beltrami eigenfunctions).
This approach generates "adaptive spectral eigenfunctions" that respond to local physical variations while maintaining global spectral properties. Mathematically, HPM serves as a generalization of existing frameworks:
- By ignoring local features, HPM simplifies to a Fixed Spectral Neural Operator.
- By ignoring the spectral basis, it becomes equivalent to a Linear Attention Mechanism.
By balancing these components, HPM captures a holistic view of the underlying physics. The model is implemented as a stack of layers featuring a multi-head design to simultaneously capture various aspects of the operator mapping.
Experimental Evaluation
HPM was evaluated across nine diverse PDE benchmarks, encompassing both structured and unstructured meshes:
- Structured Mesh: Darcy Flow, Navier-Stokes, Airfoil (irregular domain on a grid), and Plasticity.
- Unstructured Mesh: Irregular Darcy, Pipe Turbulence, Heat Transfer, Composite manufacturing, and Blood Flow in an aorta.
Key Results
- Superior Accuracy: HPM consistently outperformed state-of-the-art (SOTA) baselines. On the Navier-Stokes benchmark, HPM achieved a relative error of 7.34e-2, significantly lower than the 9.00e-2 achieved by Transolver.
- Zero-Shot Generalization: HPM demonstrated exceptional performance on unseen resolutions. When trained on a $221 \times 51$ grid and tested on a coarser $45 \times 11$ grid, HPM maintained an error of 5.34e-2, whereas the attention-based Transolver's error increased to 9.73e-2.
- Efficiency and Interpretability: HPM converges faster than pure attention models and maintains lower inference latency. Visualization of learned modulation patterns reveals that HPM naturally focuses high-order functions on regions with rapid physical changes, such as boundaries or shock waves.
Conclusion and Future Work
The Holistic Physics Mixer effectively resolves the trade-off between spectral priors and local adaptability. While the current implementation uses a simple point-wise softmax coupling, future work could explore more sophisticated, context-aware mechanisms. Additionally, the "Holistic Physics Transform" concept holds potential for broader applications in computer vision and complex time-series analysis beyond the domain of PDEs.
Summary updated