Conditionally Parameterized, Discretization-Aware Neural Networks for Mesh-Based Modeling of Physical Systems

Jiayang Xu, Aniruddhe Pradhan, Karthik Duraisamy

TL;DR

This paper introduces "Conditionally-Parameterized" (CP) neural networks that can learn to incorporate physical quantities and discretization information directly into their weights. This approach, exemplified by the CP-GNet architecture, significantly improves performance on complex physics simulations, especially on irregular meshes, and requires less training data compared to traditional methods.

What Problem Does This Paper Address?

Simulating complex physical systems often involves solving partial differential equations (PDEs) on unstructured meshes. While neural networks show promise for accelerating these simulations (e.g., surrogate modeling, reduced-order modeling), existing architectures often struggle with:

  • Ignoring physical context: They treat input features (like geometry or physical properties) as generic data, failing to leverage their inherent relationships.
  • Handling irregular meshes: Standard Convolutional Neural Networks (CNNs) require data to be on structured grids, necessitating costly and error-prone interpolation for unstructured meshes.
  • Encoding discretization information: The way PDEs are numerically discretized (e.g., finite volume methods) contains crucial information about physical processes that is typically ignored by neural networks.
  • Data inefficiency: Training robust models for complex physics often requires vast amounts of simulation data.

This paper aims to bridge this gap by developing neural network architectures that can explicitly learn from and incorporate physical quantities and numerical discretization details, leading to more efficient, accurate, and data-frugal models for mesh-based simulations.

Methods

The core innovation is Conditional Parameterization (CP). Instead of fixed weights, CP networks use trainable functions of input parameters to generate the network's weights. This allows the network to adapt its behavior based on specific physical conditions or discretization schemes.

Key aspects of their approach include:

  • CP-Dense Layers: A generalization of standard dense layers where weights are functions of input parameters. This allows for learning higher-order terms and incorporating physical quantities.
  • CP-GNet Architecture: A graph neural network (GNN) specifically designed for mesh-based simulations. It uses CP-Dense layers within its encoder, processor, and decoder.
  • Encoder: Processes node (cell) and edge (face) features, using CP-Dense layers to capture high-order interactions.
  • Processor: Employs CP message-passing (CP-MP) for flux terms and CP-Dense layers for source terms, effectively modeling discretized PDE components.
  • Decoder: Translates learned latent representations back to physical quantities.
  • Discretization-Aware Parameters: The parameters used for conditional parameterization include physical quantities (e.g., Reynolds number) and mesh topology features (e.g., cell volume, edge length, normal vectors).
  • Boundary Treatment: A novel approach using "ghost edges" to represent boundary conditions, allowing the CP-GNet to handle complex boundary types more effectively than traditional GNNs.

Datasets

The paper evaluates their methods on several challenging tasks, using data generated from simulations:

  • 1D Viscous Burgers Equation: Used for discovering closure models for coarse-grained simulations. Data is generated from high-resolution simulations and then filtered to create low-resolution counterparts.
  • Turbulent Channel Flow: Used for super-resolution tasks, recovering high-order velocity coefficients from low-order ones. Data is sourced from a public dataset.
  • Rocket Engine Injector (Combustion): A highly complex, public dataset involving reacting flows with multiple variables (pressure, velocity, species mass fractions) on a 2D finite-volume mesh. This is a key benchmark for their CP-GNet.
  • Incompressible Flow over a Cylinder: A standard benchmark for GNNs, used for comparison with MeshGraphNets. Data is from a vertex-centered FEM dataset.

Key Insights and Ideas

  • Physics-Informed Weights: The core idea of CP networks allows them to learn functional relationships that are directly tied to physical phenomena and numerical discretizations, rather than just fitting data patterns. This leads to better generalization and data efficiency.
  • Discretization as a Feature: By using discretization information (like cell geometry or differential operators) as conditional parameters, the networks can learn to mimic the behavior of numerical solvers more effectively.
  • Data Efficiency: CP networks, particularly CP-GNet, demonstrate superior performance even with significantly less training data compared to traditional GNNs or non-CP counterparts. This is crucial for expensive simulation workflows.
  • Handling Irregular Meshes: The CP-GNet architecture, combined with its boundary treatment, shows strong performance on complex, irregular meshes where standard CNNs would struggle.
  • Exact Fitting of Discretized Terms: In a simplified 2D advection-diffusion problem, CP layers were shown to be able to exactly recover the discretized PDE terms with very limited training data, highlighting their power in encoding mathematical structures.

Key Results

  • Closure Modeling: CP-CNN and CP-DDP models significantly improved accuracy (lower Mean Absolute Error) and stability compared to their non-CP counterparts for approximating closure terms in the viscous Burgers equation.
  • Super-Resolution: CP-MLP models achieved significantly reduced errors in energy spectra integrals compared to standard MLPs, indicating better reconstruction of small-scale flow structures.
  • Reacting Flow Simulation: The CP-GNet demonstrated state-of-the-art performance in predicting reacting flows on irregular meshes, maintaining high accuracy over hundreds of steps with less training data than baseline GNNs. It was also capable of standalone prediction, a novel achievement.
  • Comparison with MeshGraphNets: In reacting flow, CP-GNet showed lower RMSE and better visual fidelity to ground truth compared to MeshGraphNets. For incompressible flow over a cylinder, CP-GNet achieved comparable or better long-rollout RMSE with less training data.
  • Speedup: While not the primary focus, the paper implies significant potential for speedup in prediction compared to full-fidelity simulations, especially with the CP-GNet.

Limitations and Future Work

  • Long-Term Stability: The authors acknowledge that long-time stability guarantees remain a limitation for data-driven flow predictions, as seen in some extended predictions of the reacting flow.
  • Generalizability: While CP models show improved generalization, ensuring robustness across vastly different physical regimes or mesh configurations is an ongoing challenge.
  • Parameterization Choice: The effectiveness of CP depends on choosing appropriate physical and discretization parameters. Identifying the optimal set of parameters for diverse problems is an area for further research.
  • Computational Cost: While CP can reduce data requirements, the conditional parameterization itself can linearly increase the number of parameters. The authors suggest this can be compensated by reducing latent vector sizes, but further optimization might be needed.

Future work could focus on developing more robust long-term prediction capabilities, exploring more sophisticated parameterization strategies, and applying CP networks to an even wider range of complex physical systems and mesh types.

Summary updated

Abstract

Simulations of complex physical systems are typically realized by discretizing partial differential equations (PDEs) on unstructured meshes. While neural networks have recently been explored for surrogate and reduced order modeling of PDE solutions, they often ignore interactions or hierarchical relations between input features, and process them as concatenated mixtures. We generalize the idea of conditional parameterization -- using trainable functions of input parameters to generate the weights of a neural network, and extend them in a flexible way to encode critical information. Inspired by discretized numerical methods, choices of the parameters include physical quantities and mesh topology features. The functional relation between the modeled features and the parameters is built into the network architecture. The method is implemented on different networks and applied to frontier scientific machine learning tasks including the discovery of unmodeled physics, super-resolution of coarse fields, and the simulation of unsteady flows with chemical reactions. The results show that the conditionally-parameterized networks provide superior performance compared to their traditional counterparts. The CP-GNet - an architecture that can be trained on very few data snapshots - is proposed as the first deep learning model capable of standalone prediction of reacting flows on irregular meshes.
BibTeX
@inproceedings{xu2021conditionally,
  author = {Jiayang Xu and Aniruddhe Pradhan and Karthik Duraisamy},
  title = {{Conditionally Parameterized, Discretization-Aware Neural Networks for Mesh-Based Modeling of Physical Systems}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=0yMGEUQKd2D},
}
Figure