Data-driven discretization: machine learning for coarse graining of partial differential equations

Yohai Bar‐Sinai, Stephan Hoyer, Jason Hickey, Michael P. Brenner

Abstract

The numerical solution of partial differential equations (PDEs) is challenging because of the need to resolve spatiotemporal features over wide length and timescales. Often, it is computationally intractable to resolve the finest features in the solution. The only recourse is to use approximate coarse-grained representations, which aim to accurately represent long-wavelength dynamics while properly accounting for unresolved small scale physics. Deriving such coarse grained equations is notoriously difficult, and often ad hoc. Here we introduce data driven discretization, a method for learning optimized approximations to PDEs based on actual solutions to the known underlying equations. Our approach uses neural networks to estimate spatial derivatives, which are optimized end-to-end to best satisfy the equations on a low resolution grid. The resulting numerical methods are remarkably accurate, allowing us to integrate in time a collection of nonlinear equations in one spatial dimension at resolutions 4-8x coarser than is possible with standard finite difference methods.

Introduction

Many systems in physics and engineering—from weather patterns to fluid flow—are described by nonlinear PDEs. Solving these numerically requires discretization, or breaking the continuous world into a grid of points. The "catch-22" of numerical simulation is resolution: capturing fine-scale features like shock waves or turbulence requires a very fine grid, which is computationally expensive. Conversely, coarse grids often fail to account for "sub-grid" physics, leading to significant errors or simulation instability. Traditionally, researchers have relied on manually derived "effective theories" or "closure models," which are often ad hoc and difficult to generalize.

Data-Driven Discretization

The proposed method, Data-Driven Discretization, replaces fixed, human-derived formulas for spatial derivatives with a neural network that "reconstructs" missing fine-scale information.

Technical Framework

The architecture incorporates several key components to ensure both flexibility and physical consistency:

  • Pseudo-linear Representation: The network predicts coefficients ($\alpha_i$) for a generalized finite-difference formula rather than direct values. This allows the model to adapt to local features like shocks while remaining interpretable.
  • Physical Constraints: Conservation laws are enforced via a Finite Volume Method, where the network predicts fluxes at cell boundaries. Additionally, Taylor expansions constrain coefficients to ensure at least first-order polynomial accuracy.
  • End-to-End Optimization: Training minimizes the error in the time derivative of the field, ensuring the learned scheme remains stable for long-term integration.

Experimental Results

The authors evaluated the method on three canonical nonlinear equations: Burgers’ (fluid dynamics/shocks), Korteweg-de Vries (solitary waves), and Kuramoto-Sivashinsky (chaotic systems).

Spatiotemporal evolution and error analysis comparing neural network discretization to standard methods across various resample factors.

As shown in the spatiotemporal comparisons in Figure 3, the neural network maintains structural integrity at resolutions where standard first-order or WENO schemes fail or produce significant artifacts. Quantitatively, the mean absolute error for the neural network at high resample factors (e.g., 8x or 16x) is often orders of magnitude lower than traditional baselines.

Performance and Generalization

A key metric for simulation success is the "valid simulation time"—the duration for which the numerical solution remains within a specified error tolerance of the ground truth.

Median valid simulation time across different resample factors for Burgers', KdV, and KS equations, demonstrating the superior stability of the neural network approach.

As illustrated in Figure 4, the neural network approach consistently achieves higher valid simulation times at coarser resolutions (higher resample factors) compared to standard methods. For instance, in Burgers' equation, the model achieves an 8x efficiency gain, maintaining accuracy on a grid 8 times coarser than required by standard methods. Furthermore, models trained on small spatial domains ($2\pi$) successfully generalized to domains 10x larger ($20\pi$), indicating that the network learned local physical rules rather than memorizing specific global solutions.

Conclusion and Future Work

This work demonstrates a "hybrid intelligence" approach, combining the flexibility of machine learning with the stability of classical numerical frameworks. While the current implementation faces computational overhead due to neural network inference at each step, future optimizations like "banks of linear filters" could mitigate this. Expanding this framework to 2D and 3D simulations and irregular grids remains a primary objective for future research.

Summary updated

Abstract

The numerical solution of partial differential equations (PDEs) is challenging because of the need to resolve spatiotemporal features over wide length and timescales. Often, it is computationally intractable to resolve the finest features in the solution. The only recourse is to use approximate coarse-grained representations, which aim to accurately represent long-wavelength dynamics while properly accounting for unresolved small scale physics. Deriving such coarse grained equations is notoriously difficult, and often \emph{ad hoc}. Here we introduce \emph{data driven discretization}, a method for learning optimized approximations to PDEs based on actual solutions to the known underlying equations. Our approach uses neural networks to estimate spatial derivatives, which are optimized end-to-end to best satisfy the equations on a low resolution grid. The resulting numerical methods are remarkably accurate, allowing us to integrate in time a collection of nonlinear equations in one spatial dimension at resolutions 4-8x coarser than is possible with standard finite difference methods.
BibTeX
@inproceedings{barsinai2019data,
  author = {Yohai Bar‐Sinai and Stephan Hoyer and Jason Hickey and Michael P. Brenner},
  title = {{Data-driven discretization: machine learning for coarse graining of partial differential equations}},
  booktitle = {Proceedings of the National Academy of Sciences},
  year = {2019},
  doi = {10.1073/pnas.1814058116},
}
Figure