Graph Neural PDE Solvers with Conservation and Similarity-Equivariance

Masanobu Horie, Naoto Mitsume

Abstract

Utilizing machine learning to address partial differential equations (PDEs) presents significant challenges due to the diversity of spatial domains and their corresponding state configurations, which complicates the task of encompassing all potential scenarios through data-driven methodologies alone. Moreover, there are legitimate concerns regarding the generalization and reliability of such approaches, as they often overlook inherent physical constraints. In response to these challenges, this study introduces a novel machine-learning architecture that is highly generalizable and adheres to conservation laws and physical symmetries, thereby ensuring greater reliability. The foundation of this architecture is graph neural networks (GNNs), which are adept at accommodating a variety of shapes and forms. Additionally, we explore the parallels between GNNs and traditional numerical solvers, facilitating a seamless integration of conservative principles and symmetries into machine learning models. Our findings from experiments demonstrate that the model's inclusion of physical laws significantly enhances its generalizability, i.e., no significant accuracy degradation for unseen spatial domains while other models degrade.

Introduction

Simulating complex physical systems—such as fluid dynamics or heat transfer—traditionally relies on solving Partial Differential Equations (PDEs) using numerical methods like the Finite Volume Method (FVM). While reliable and physically consistent, these solvers are often computationally intensive. Recent advancements in machine learning, particularly Graph Neural Networks (GNNs), offer a faster alternative. However, standard GNNs frequently suffer from two critical flaws: they violate fundamental conservation laws (e.g., mass or energy conservation) and fail to generalize to spatial domains with different scales or rotations than those seen during training.

This paper introduces FluxGNN, a physics-aware architecture designed to bridge the gap between the expressivity of GNNs and the physical rigor of FVM. By embedding conservation laws and similarity-equivariance directly into the model's structure, FluxGNN achieves high accuracy and robust spatial extrapolation.

Methodology: The FluxGNN Architecture

FluxGNN reinterprets the message-passing mechanism of GNNs through the lens of numerical flux in FVM. The core innovation lies in treating GNN messages as physical fluxes passing through the faces of a mesh.

Conceptual comparison of GNN, FVM, and the proposed FluxGNN architecture.

Locally Conservative GNNs

To ensure local conservation, the authors mathematically demonstrate that the message-passing function must be anti-symmetric ($m_{ij} = -m_{ji}$) and the update function must be linear (effectively an identity map). This ensures that any quantity leaving one cell is exactly accounted for as entering the adjacent cell, preventing the "disappearing mass" problem common in standard ML models.

Similarity-Equivariance

Physical laws are consistent across scales and rotations. FluxGNN achieves similarity-equivariance—consistency across translation, rotation, reflection, and scaling—by utilizing dimensionless inputs (such as the Courant number) and specialized Multi-Layer Perceptron (MLP) layers that handle vector magnitudes and directions consistently. This allows the model to learn underlying physical principles rather than memorizing specific spatial patterns.

Experimental Evaluation

The model was evaluated on two primary benchmarks: the 1D Convection–Diffusion equation and the 2D Navier–Stokes equations involving fluid mixtures. To test generalization, the researchers utilized "out-of-domain" datasets featuring rotated geometries, scaled domains, and "taller" shapes significantly different from the training set.

Results and Discussion

FluxGNN demonstrated superior performance across several metrics:

  • Conservation Accuracy: While baseline models like MP-PDE and PENN exhibited significant conservation errors, FluxGNN maintained errors near machine precision ($10^{-7}$ to $10^{-8}$).
  • Spatial Extrapolation: On unseen, taller geometries where baseline models often diverged or produced numerical errors (NaNs), FluxGNN generalized gracefully, maintaining high predictive accuracy.
  • Predictive Fidelity: As shown in the comparison of state variables, FluxGNN closely tracks the ground truth, outperforming traditional FVM on coarse grids by learning more effective numerical fluxes.
Comparison of FluxGNN predictions against ground truth and traditional FVM solvers.

Conclusion

FluxGNN represents a significant step toward reliable, physics-informed machine learning for PDE discovery and simulation. By enforcing physical constraints as inductive biases, the model avoids unphysical solutions and generalizes to complex, unseen geometries. Future work will explore the integration of "slope limiters" to enhance long-term stability and apply the architecture to broader classes of PDEs, such as traffic flow and optimal transport.

Summary updated

Abstract

Utilizing machine learning to address partial differential equations (PDEs) presents significant challenges due to the diversity of spatial domains and their corresponding state configurations, which complicates the task of encompassing all potential scenarios through data-driven methodologies alone. Moreover, there are legitimate concerns regarding the generalization and reliability of such approaches, as they often overlook inherent physical constraints. In response to these challenges, this study introduces a novel machine-learning architecture that is highly generalizable and adheres to conservation laws and physical symmetries, thereby ensuring greater reliability. The foundation of this architecture is graph neural networks (GNNs), which are adept at accommodating a variety of shapes and forms. Additionally, we explore the parallels between GNNs and traditional numerical solvers, facilitating a seamless integration of conservative principles and symmetries into machine learning models. Our findings from experiments demonstrate that the model's inclusion of physical laws significantly enhances its generalizability, i.e., no significant accuracy degradation for unseen spatial domains while other models degrade. The code is available at https://github.com/yellowshippo/fluxgnn-icml2024.
BibTeX
@inproceedings{horie2024graph,
  author = {Masanobu Horie and Naoto Mitsume},
  title = {{Graph Neural PDE Solvers with Conservation and Similarity-Equivariance}},
  booktitle = {International Conference on Machine Learning},
  year = {2024},
  url = {https://openreview.net/forum?id=WajJf47TUi},
}
Figure