Resolution Invariant Neural Operators for Continuous Physical Systems

243 concepts197 papers1 proposal

When we use deep learning to simulate physical systems—such as fluid dynamics, plasma turbulence, or structural strain—we face a fundamental mismatch. The physical world is continuous, governed by partial differential equations (PDEs) that exist in infinite-dimensional function spaces. Yet, standard neural networks process data as discrete arrays: a grid of pixels, a matrix of sensor readings, or a fixed simulation mesh.

This creates a severe limitation known as discretization sensitivity. As discussed by Zongyi Li et al. in their work on Multipole Graph Neural Operator for Parametric Partial Differential Equations, models trained on a specific grid resolution often fail catastrophically when evaluated on a finer or coarser mesh. They lack the ability to perform "zero-shot super-resolution" or adapt to irregular geometries without complete retraining.

To build neural surrogates that truly understand physics, we must design architectures that are discretization-invariant—meaning they learn the underlying continuous operator rather than a grid-specific mapping.


Why Standard Networks Fail: The Bitmap Analogy

To understand why standard architectures break down, consider the difference between a bitmap image and a vector graphic. A bitmap is defined by a fixed grid of pixels; if you zoom in, it becomes pixelated and loses structure. A vector graphic, by contrast, is defined by continuous mathematical formulas (lines, curves, and shapes) and can be rendered perfectly at any resolution.

Standard Convolutional Neural Networks (CNNs) are "bitmap" models. A convolutional filter learns weights for a fixed spatial neighborhood (e.g., a $3 \times 3$ patch of pixels). If you double the resolution of your simulation grid, that $3 \times 3$ filter now covers only a quarter of the physical area it did during training. The model's learned physics are fundamentally tied to the grid scale.

As Marc Anton Finzi et al. point out in Probabilistic Numeric Convolutional Neural Networks, discrete operators ignore the underlying discretization error of the continuous signal, leading to a performance collapse when sampling frequencies shift.


The Solution: Mapping Functions to Functions

To achieve resolution independence, the scientific machine learning community has shifted from learning functions (which map discrete vectors to discrete vectors) to learning operators (which map continuous input functions to continuous output functions).

As Nicholas H. Nelsen et al. demonstrate in Operator Learning Using Random Features: A Tool for Scientific Computing, framing the problem in infinite-dimensional function spaces ensures that the model's predictions remain mathematically consistent regardless of how finely the domain is discretized.

Modern architectures achieve this through three primary mechanisms:

1. Coordinate-Based Queries and Neural Fields

One powerful way to bypass fixed grids is to decouple the representation of the physical field from the underlying mesh. Instead of predicting an entire grid at once, models use coordinate-based queries.

For example, Jan Hagnberger et al. developed Vectorized Conditional Neural Fields: A Framework for Solving Time-dependent Parametric Partial Differential Equations, which process multiple spatio-temporal query points in parallel and explicitly model their physical dependencies via attention mechanisms. Similarly, Yuan Yin et al. introduced DINO in Continuous PDE Dynamics Forecasting with Implicit Neural Representations, combining Implicit Neural Representations (INRs) with Latent ODEs to forecast continuous fields.

Because these models can be queried at any continuous coordinate $(x, y, z, t)$, zero-shot super-resolution becomes a natural property of the architecture rather than an engineering challenge.

2. Continuous Kernel Integrals

Instead of discrete matrix multiplications, resolution-invariant models often use kernel integral operators of the form:

$$G(u)(x) = \int_{\mathcal{D}} \kappa(x, y) u(y) dy$$

where the kernel $\kappa(x, y)$ is parameterized by a neural network. Because integration is a continuous operation, it can be approximated on any discrete grid using standard numerical quadrature (like Riemann sums).

To make this computationally efficient, models often operate in transformed spaces. Peiyan Hu et al. proposed the Wavelet Diffusion Neural Operator, which performs diffusion-based generative modeling in the wavelet domain. This allows the model to capture sharp physical discontinuities (like shocks) while generalizing seamlessly to higher spatial resolutions than those seen during training.

3. Mesh-Agnostic Graphs and Transformers

When dealing with complex or irregular engineering geometries, graph-based representations offer an elegant alternative to rigid grids. By treating simulation nodes as vertices and physical interactions as edges, Graph Neural Networks (GNNs) can naturally handle varying mesh densities.

Andrey Bryutkin et al. leveraged this in HAMLET: Graph Transformer Neural Operator for Partial Differential Equations to maintain discretization invariance across irregular shapes. To push this further, Hoyun Choi et al. integrated GNNs with classical numerical integration in GNRK: Graph Neural Runge-Kutta method for solving partial differential equations, creating a solver that is inherently resolution-agnostic across different time steps and network topologies.


Bridging the Gap with Classical Solvers

A major benefit of resolution invariance is the ability to hybridize deep learning with classical numerical methods. Because the neural operator remains valid at high resolutions, it can be embedded directly into traditional engineering workflows.

In Neural operators meet conjugate gradients: The FCG-NO method for efficient PDE solving, Alexander Rudikov demonstrates how a resolution-invariant neural operator can act as a nonlinear preconditioner for the Flexible Conjugate Gradient (FCG) method. This hybrid approach retains the exact mathematical guarantees of classical solvers while drastically reducing computation time on high-fidelity grids that the neural network never saw during training.

Similarly, Yue Qiu shows in Resolution invariant deep operator network for PDEs with complex geometries that models trained on cheap, low-resolution data can successfully predict high-resolution solutions across entirely different spatial domains, bypassing the need for expensive data collection.


Nuance and Open Challenges

While the mathematical promise of resolution invariance is clear, achieving it perfectly in practice remains an active area of research.

As summarized by Mathias Oster et al. in Mini-Workshop: Nonlinear Approximation of High-dimensional Functions in Scientific Computing, empirical evidence suggests that many models claiming discretization invariance are still implicitly tied to their training grids due to subtle architectural choices, such as boundary padding or fixed frequency cutoffs.

Furthermore, Huayu Deng notes in EvoMesh: Adaptive Physical Simulation with Hierarchical Graph Evolutions that building a completely scale-invariant representation of chaotic multiscale physics—where new high-frequency dynamics emerge at finer scales—is an ongoing challenge.

Nevertheless, moving away from grid-dependent architectures toward continuous operator learning represents a foundational shift, unlocking the ability to deploy AI models across diverse, multi-scale physical simulations without retraining.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_discretization_sensitivity_03a88995,
  title = {discretization sensitivity},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/discretization-sensitivity/rdr_03a889953d4e4d88ae4e88cae9c650f6},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_discretization_sensitivity_03a88995_v1,
  title = {discretization sensitivity},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/discretization-sensitivity/rdr_03a889953d4e4d88ae4e88cae9c650f6/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake