Holistic Physics Solver: Learning PDEs in a Unified Spectral-Physical Space

Xihang Yue, Yi Yang, Linchao Zhu

TL;DR The Holistic Physics Mixer (HPM) bridges the gap between the two dominant paradigms in neural PDE solvers: spectral methods and attention mechanisms. By integrating physical states and spectral basis functions into a unified "holistic" space, HPM achieves the local flexibility of Transformers and the global generalization of Fourier Neural Operators, outperforming state-of-the-art models in both accuracy and efficiency.

The Dichotomy in Neural PDE Solvers

Partial Differential Equations (PDEs) are the bedrock of physics and engineering, but traditional numerical solvers are computationally expensive. Recently, Neural Operators have emerged as a fast, data-driven alternative. However, the field has historically been split into two camps:

  1. Spectral-based methods (e.g., FNO): These use fixed spectral bases (like Fourier transforms) to enforce global continuity. They generalize well to different grid resolutions but struggle to capture fine-scale, local variations because their "filters" are applied globally and uniformly.
  2. Attention-based methods (e.g., Transformers): These are highly flexible and can adapt to local features. However, they lack physical priors, require massive amounts of data, and often fail in "zero-shot" scenarios when tested on resolutions different from their training data.

This creates a strict trade-off: solvers typically offer either global stability or local adaptability, but rarely both.

Comparison of Spectral, Attention, and Holistic Physics Mixers

The Holistic Physics Mixer (HPM)

To resolve this trade-off, the authors introduce the Holistic Physics Mixer (HPM). Instead of choosing between a fixed spectral basis or a purely data-driven attention matrix, HPM uses a coupling function $H(x, \Phi)$ to integrate both.

  • The Coupling Mechanism: HPM takes the local physical features ($x$) and uses them to modulate a preset spectral basis ($\Phi$). Mathematically, this is often implemented as: $$H(x, \Phi) = \text{Softmax}(\text{MLP}(x)) \odot \Phi$$
  • Adaptive Eigenfunctions: This approach creates "adaptive spectral eigenfunctions." The model learns to tune its frequency response at every single point in space based on the local physics, rather than applying a blanket frequency filter across the entire domain.
  • Unified Framework: HPM acts as a generalized framework. If the modulation is fixed, it recovers traditional spectral methods. If the spectral basis is ignored, it behaves like a linear attention mechanism.
  • Geometric Flexibility: To handle irregular shapes like airfoils or blood vessels, HPM uses Laplace-Beltrami Operator (LBO) eigenfunctions as its basis, allowing it to operate seamlessly on both structured grids and unstructured meshes.
Architecture of the Holistic Physics Mixer and its coupling mechanism

Experimental Validation

The authors evaluated HPM across a rigorous battery of 9 different PDE problems, encompassing both structured meshes (e.g., Navier-Stokes fluid dynamics, steady-state Darcy Flow, Airfoil aerodynamics) and unstructured meshes (e.g., Pipe Turbulence, Blood Flow in an aorta, Heat Transfer).

Key Insights and Results

  • State-of-the-Art Performance: HPM consistently outperformed all baselines. On the Navier-Stokes benchmark, HPM achieved a relative error of 7.34e-2, compared to 9.00e-2 for the previous leading model, Transolver.
  • Zero-Shot Generalization: HPM inherits the resolution invariance of spectral methods. When trained on a $221 \times 51$ resolution and tested on a much coarser $45 \times 11$ grid, HPM’s error remained low (5.34e-2), while Transolver’s error spiked to 9.73e-2.
  • Spatially Adaptive Modulation: Visualizations reveal that HPM naturally learns to use high-frequency components near boundaries and areas of sharp change, while relying on low-frequency components for smooth regions.
  • Efficiency and Speed: By using a simple point-wise coupling, HPM avoids the $O(N^2)$ quadratic complexity of standard self-attention. It is roughly 1.7x faster than Transolver and uses significantly fewer parameters (0.75M vs 2.8M), making it highly practical for real-time engineering applications.
  • Data Efficiency: HPM matched the performance of competing models using only 200 training trajectories, whereas others required up to 1,000.

Limitations and Future Work

While the Softmax-based coupling is highly effective, it is a fundamental design that does not yet explicitly encode specific physical constraints like conservation laws. Furthermore, scaling the framework from 2D or simplified 3D manifolds to massive, high-fidelity 3D industrial simulations remains a future challenge. Ultimately, the "Holistic Physics Transform" concept shows promise not just for PDEs, but potentially for broader computer vision or time-series tasks where balancing global structure and local detail is critical.

Summary updated

Abstract

Recent advances in operator learning have produced two distinct approaches for solving partial differential equations (PDEs): attention-based methods offering point-level adaptability but lacking spectral constraints, and spectral-based methods providing domain-level continuity priors but limited in local flexibility. This dichotomy has hindered the development of PDE solvers with both strong flexibility and generalization capability. This work introduces Holistic Physics Mixer (HPM), a novel framework that bridges this gap by integrating spectral and physical information in a unified space. HPM unifies both approaches as special cases while enabling more powerful spectral-physical interactions beyond either method alone. This enables HPM to inherit both the strong generalization of spectral methods and the flexibility of attention mechanisms while avoiding their respective limitations. Through extensive experiments across diverse PDE problems, we demonstrate that HPM consistently outperforms state-of-the-art methods in both accuracy and computational efficiency, while maintaining strong generalization capabilities with limited training data and excellent zero-shot performance on unseen resolutions.
BibTeX
@inproceedings{yue2025holistic,
  author = {Xihang Yue and Yi Yang and Linchao Zhu},
  title = {{Holistic Physics Solver: Learning PDEs in a Unified Spectral-Physical Space}},
  booktitle = {International Conference on Machine Learning},
  year = {2025},
  url = {https://openreview.net/forum?id=oB5a6yIAmF},
}
Figure