Multi-Scale Architectures for Physics Informed Neural Operators

This direction is no longer present in the current build. Its last version stays readable and citable here.

Version 1 of 150 papers51 conceptsfrozen 2026-10-04

In physical systems, the "whole" is rarely just the sum of its parts. A storm system is driven by planetary-scale pressure gradients, yet its most destructive power resides in localized, high-frequency turbulence. Simulating these systems requires capturing a "multi-scale" reality: global patterns that dictate the environment and local details that drive the dynamics.

Standard neural networks often suffer from a form of "tunnel vision." If they focus on the global context, they blur out critical local features like shock waves or eddies. If they focus on local details, they lose the long-range dependencies that maintain physical consistency across the entire system. Researchers are now developing architectures that explicitly decouple and then fuse these disparate scales.

The Conflict of Local Precision and Global Context

The fundamental challenge in physics-informed machine learning is that different physical variables equilibrate at different rates and spatial scales. In subsurface fluid dynamics, for instance, pressure changes propagate globally across a reservoir, while fluid saturation is dictated by local rock properties.

As Tailin Wu and colleagues point out in Learning Large-scale Subsurface Simulations with a Hybrid Graph Network Simulator, a single-scale model cannot efficiently bridge this gap. If you use a high-resolution grid everywhere to capture local details, the computational cost becomes prohibitive. If you use a coarse grid, you miss the "physics" happening at the micro-scale.

To solve this, researchers like Jan-Willem van de Meent have proposed Multi-Scale Attention (MSPT), which allows a model to simultaneously account for localized material stress and global pressure changes. The intuition is to treat the simulation like a camera with both a wide-angle and a macro lens, processing both signals in parallel.

Decoupling Scales via Hybrid Architectures

One of the most successful strategies for handling multi-scale physics is the hybrid architecture. These models assign different neural mechanisms to different physical scales:

  1. Local Specialists: Graph Neural Networks (GNNs) or small-kernel Convolutions (CNNs) are used to model immediate interactions, such as collisions between particles or fluid flow between adjacent cells.
  2. Global Generalists: Fourier Neural Operators (FNOs), State Space Models (Mamba), or Transformers capture long-range dependencies, such as gravity or planetary-scale winds.

For example, Honggang Jia and Nan Chen introduced RadioMamba, which combines the global modeling power of State Space Models with the local precision of CNNs. Similarly, Yutao Du developed the Global-local Fourier Neural Operator (GL-FNO) to simulate solar magnetic fields. By using a dual-branch approach, the model avoids the "blurring" effect common in global-only operators while maintaining the 20,000x speedup over traditional solvers.

Hierarchical and Multi-Grid Schemes

Another approach borrows from classical numerical analysis: the multi-grid method. In traditional solvers, errors are reduced by iterating across different grid resolutions—coarse grids smooth out global errors, while fine grids resolve local details.

Léon Migus and Patrick Gallinari implemented this in Multi-scale Physical Representations for Approximating PDE Solutions with Graph Neural Operators by using F-cycle and W-cycle schemes. This hierarchical structure allows the network to pass information "up and down" the scale ladder.

Jean Kossaifi and Anima Anandkumar extended this with MG-TFNO, which uses domain decomposition and low-rank tensors to handle high-resolution turbulent fluid dynamics. The hierarchy ensures that the model doesn't just "see" the turbulence; it understands how the large-scale energy cascade feeds into the smallest vortices.

Spectral Fusion: Fourier meets Wavelets

The choice of mathematical transform also dictates how well a model handles scale. The Fourier Transform is excellent at capturing global, periodic patterns but struggles with "sharp" local changes like shock waves.

To fix this, Chenhong Zhou and Jie Chen proposed the Spectral Attention Operator Transformer (SAOT). SAOT combines:

  • Fourier Transforms for long-range, smooth dependencies.
  • Wavelet Transforms for localized, abrupt changes.

By fusing these in a gated mechanism, the model can "switch" its attention between the horizon and the immediate vicinity, depending on the physical requirements of the moment.

Toward Scale-Consistent Learning

A final, emerging frontier is scale consistency. Most neural networks are "resolution-dependent"—a model trained on a small pipe might fail on a large ocean because it doesn't realize the underlying physics is the same.

Zongyi Li and colleagues argue in Scale-Consistent Learning for Partial Differential Equations that models must be designed to recognize that physical laws remain consistent across scales. This involves building inductive biases into the architecture so that the model doesn't just memorize a specific resolution but learns the "operator" that governs the system at any scale.

As Fan Zhang notes in Turb-L1, failing to track these simultaneous interactions across scales is why many models "collapse" during long-term simulations. By explicitly modeling the multi-scale energy cascade, researchers are finally moving toward neural solvers that are as stable and precise as the physical laws they aim to emulate.

Papers in this version

History & citationVersion 1 · 2026-10-04

No longer present in the current build · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_multi_scale_architectures_for_physics_in_5e5f2ab6,
  title = {Multi-Scale Architectures for Physics Informed Neural Operators},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/multi-scale-architectures-for-physics-informed-neural-operators/rdr_5e5f2ab6abe5470aa99cdb40c06e90e7},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_multi_scale_architectures_for_physics_in_5e5f2ab6_v1,
  title = {Multi-Scale Architectures for Physics Informed Neural Operators},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/multi-scale-architectures-for-physics-informed-neural-operators/rdr_5e5f2ab6abe5470aa99cdb40c06e90e7/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}