Abstract
Neural PDE solvers offer significant speedups over traditional numerical methods but often struggle with geometric aliasing and high computational costs on large-scale meshes. We present PGOT (Physics-Geometry Operator Transformer), an architecture that achieves state-of-the-art accuracy on complex 3D simulations. PGOT introduces SpecGeo-Attention for multi-scale geometry injection and TaylorDecomp-FFN for dynamic routing between linear and non-linear physical regimes. Our model maintains linear computational scaling while significantly reducing error on industrial-grade benchmarks.
Introduction: The Scalability Barrier
In scientific computing, neural operators serve as fast alternatives to traditional Finite Element Analysis. However, standard Transformers suffer from $O(N^2)$ complexity, making them impractical for simulations with millions of mesh points. While "efficient" Transformers reduce this to $O(N)$ through token aggregation, they often introduce Geometric Aliasing. This spatial low-pass filtering blurs critical high-frequency details, such as sharp boundary layers or complex surface geometries, which are essential for accurate physical modeling.
Methods: Preserving Geometry and Physics
PGOT addresses these challenges through two core innovations integrated into the PhysGeoBlock.
SpecGeo-Attention
To combat geometric aliasing, PGOT utilizes Spectrum-Preserving Geometric Attention (SpecGeo-Attention). Unlike models that treat geometry as a static input, PGOT employs a Spectrum Encoder to generate multi-scale geometric encodings. These features are injected at every layer, ensuring the model retains high-frequency mesh details throughout the aggregation and reconstruction phases.
TaylorDecomp-FFN
Physical phenomena are rarely uniform; smooth regions are simpler to model than chaotic shock waves. Inspired by Taylor series expansions, the TaylorDecomp-FFN replaces standard feed-forward networks with a dual-expert system:
- Linear Expert: Handles smooth, low-order conservation dynamics.
- Non-linear Expert: Resolves sharp gradients and discontinuities. A spatial gating mechanism dynamically routes data based on local coordinates, optimizing "computational brainpower" for the most complex regions of the simulation.
Empirical Results
PGOT was evaluated across a variety of benchmarks, including 2D elasticity and 3D automotive aerodynamics (Shape-Net Car).
Performance and Efficiency
As shown in Figure 1, PGOT consistently outperforms existing state-of-the-art models like Transolver and SAOT. Key results include:
- Accuracy: Error reductions of 7.7% to 12.7% on standard benchmarks and up to 81.3% on surface fields in high-fidelity RANS simulations (AirfRANS).
- Scalability: PGOT maintains linear scaling, operating 4.2x faster than ONO at large mesh sizes ($N \approx 32k$).
- Robustness: The model demonstrates strong out-of-distribution generalization, accurately predicting physics under unseen conditions (e.g., varying Reynolds numbers).
Conclusion
PGOT proves that linear efficiency does not require sacrificing boundary accuracy. By decoupling geometric representation from physical dynamics, it provides a robust framework for large-scale, high-fidelity scientific simulations. Future work will explore scaling this paradigm toward multi-physics foundation models.
Summary updated