Neural Partial Differential Equation Solvers for Time Dependent Systems

This direction is merged into another direction: extension to time-dependent problems. Its last version stays readable and citable here.

Version 1 of 152 papers52 conceptsfrozen 2026-10-04

Traditional numerical methods for solving Partial Differential Equations (PDEs), such as Finite Element Analysis (FEA), have long been the backbone of engineering. However, these methods suffer from the "curse of dimensionality"—as the number of variables increases, the computational cost grows exponentially. A new frontier in Scientific Machine Learning (SciML) seeks to replace these rigid grids with neural networks that can learn the underlying physics of a system.

While researchers have achieved significant success in solving "steady-state" (elliptic) equations, where a system has reached equilibrium, the current challenge lies in capturing "transient dynamics"—how systems evolve over time.

Bypassing the Curse of Dimensionality

The primary intuition behind using neural networks for PDEs is their ability to approximate high-dimensional functions efficiently. Jianfeng Lu, Yulong Lu, and Ziang Chen have provided theoretical guarantees for this in On the Representation of Solutions to Elliptic PDEs in Barron Spaces. They demonstrate that for certain classes of equations, neural networks can approximate solutions with a complexity that scales only polynomially with dimension, rather than exponentially.

This efficiency allows models like the Physics-Informed Neural Network (PINN) to solve equations without a mesh. By encoding physical laws directly into the loss function—as seen in the work on heat transport by Amir Barati Farimani and Peter K. Eastman in Weakly-Supervised Deep Learning of Heat Transport via Physics Informed Loss—the network learns to satisfy the PDE at every point in the domain simultaneously.

From Snapshots to Operators

A major limitation of early neural solvers was that they had to be retrained for every new boundary condition or source term. To solve this, Zongyi Li, Andrew M. Stuart, and colleagues introduced the Neural Operator.

Instead of learning a single solution (a snapshot), a Neural Operator learns the mapping between infinite-dimensional function spaces. This makes the model "discretization-invariant": you can train it on a low-resolution simulation and it will still produce accurate results on a high-resolution grid. This is achieved by treating the network layers as integral operators, effectively learning the "Green’s function" or the fundamental response of the physical system, a concept further explored by Nicolas Boullé and Alex Townsend in Data-driven discovery of Green’s functions.

The Temporal Wall and Spectral Bias

Despite these advances, a "reality check" provided by Changfan Yang and Zeke Xie in the Multiphysics Bench reveals that current models struggle significantly with time-dependent dynamics.

The intuition for this failure is twofold:

  1. Causality: In steady-state problems, every point influences every other point simultaneously. In time-dependent problems, the state at time $t$ must strictly cause the state at $t+1$. Standard architectures often fail to respect this temporal flow.
  2. Spectral Bias: Neural networks naturally prefer learning low-frequency, smooth functions. However, many time-evolving systems (like turbulence or shocks) contain high-frequency details. Darui Zhao and Xiangsheng Hu address this in xLSTM-PINN by using memory-gated blocks to capture these steep gradients and rapid changes over time.

Hybridizing for Robustness

To bridge the gap between the speed of AI and the rigor of classical math, researchers are developing hybrid solvers. Alexander Rudikov and Ekaterina Muravleva developed GMsFEM-NO, which uses Neural Operators to predict the localized basis functions for the Generalized Multiscale Finite Element Method. This achieves a 60x speedup by letting the neural network handle the expensive "offline" part of the math while the "online" simulation remains grounded in traditional finite element theory.

Similarly, Yue Wu and Cuiyu He introduced XNODE-WAN, which uses "Extended Neural ODEs" to treat time as a continuous evolution rather than just another spatial dimension. This allows the model to scale to 64 dimensions while maintaining the distinct mathematical properties of parabolic equations.

The field is currently moving toward architectures that can act as "linear propagators"—models that don't just predict what a system looks like, but understand the fundamental laws of how it moves from one moment to the next.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_neural_partial_differential_equation_sol_0e940da8,
  title = {Neural Partial Differential Equation Solvers for Time Dependent Systems},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/neural-partial-differential-equation-solvers-for-time-dependent-systems/rdr_0e940da810ba4c8db8f7818afc25e67e},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_neural_partial_differential_equation_sol_0e940da8_v1,
  title = {Neural Partial Differential Equation Solvers for Time Dependent Systems},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/neural-partial-differential-equation-solvers-for-time-dependent-systems/rdr_0e940da810ba4c8db8f7818afc25e67e/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}