Learning metriplectic systems and other bracket-based dynamics

Anthony Gruber, Kookjin Lee, Haksoo Lim, Noseong Park, Nathaniel Trask2024 · DOI

Introduction

In the evolving landscape of physics-informed machine learning, a significant challenge remains: how to model dynamical systems that are not only complex but also strictly adhere to the fundamental laws of thermodynamics. Standard deep learning approaches, such as Neural Ordinary Differential Equations (NODEs), often operate as "black boxes," potentially predicting physically impossible behaviors like spontaneous energy gain or heat flowing from cold to hot.

Presented at the CMS Summer Meeting in June 2024, Anthony Gruber of Sandia National Laboratories introduced a novel framework to address these shortcomings. By leveraging Neural Metriplectic Systems (NMS) and Bracket-Based Graph Neural Networks (GNNs), this research embeds physical constraints—specifically energy conservation and entropy production—directly into the architecture of neural networks.

Title slide of the presentation by Anthony Gruber at the CMS Summer Meeting.

The Metriplectic Formalism

While Hamiltonian Neural Networks have successfully modeled conservative systems where energy is preserved, most real-world applications involve dissipation, such as friction or heat loss. Modeling these requires the metriplectic formalism, which accounts for both the First Law (energy conservation) and the Second Law (entropy generation) of thermodynamics.

The core dynamics are parameterized as: $$\dot{\mathbf{x}} = \mathbf{L}\nabla E + \mathbf{M}\nabla S$$ where $E$ represents energy and $S$ represents entropy. To ensure physical validity, the matrices $\mathbf{L}$ (skew-symmetric) and $\mathbf{M}$ (symmetric positive semi-definite) must satisfy "degeneracy" conditions: $\mathbf{L}\nabla S = 0$ and $\mathbf{M}\nabla E = 0$.

Exterior Algebra Parameterization

A key innovation in this work is the use of Exterior Algebra (EA) to factor these tensors. By using bivector fields, the authors "hard-constrain" the model to be metriplectic by construction. This geometric approach not only guarantees physical consistency but also improves computational scaling to $O(n^2)$, a significant improvement over previous $O(n^3)$ methods.

The research was developed at the Center for Computing Research (CCR).

Extending to Graphs: Bracket-Based GNNs

The research further extends these concepts to graph structures using Graph Exterior Calculus (GEC). This allows GNN layers to be interpreted as physical evolutions, such as diffusion or gradient flows, occurring on a graph.

A notable insight from this framework is the reinterpretation of standard Graph Attention Networks (GAT). The authors demonstrate that GATs can be viewed as a specific type of anisotropic heat flow (a Metric Laplacian). By framing GNNs as bracket-based dynamics, new architectures like the "Double Bracket" GNN can be designed, offering improved performance on traditional machine learning tasks by leveraging physical inductive biases.

Experimental Results

The models were rigorously evaluated across three domains:

  1. Thermodynamic Toy Models: In a "two gas containers" system, the NMS model significantly outperformed standard NODEs and previous state-of-the-art models in long-term trajectory prediction.
  2. Robotics Simulations: Using MuJoCo environments (HalfCheetah, Hopper, Swimmer), bracket-based models achieved approximately half the relative error compared to standard Neural ODE and Autoencoder baselines.
  3. Graph Benchmarks: The "Double Bracket" GNN variant outperformed standard GAT and Graph Diffusion Equations (GDE) on several node classification benchmarks (e.g., Cora, CiteSeer, PubMed), proving that physics-informed structures enhance general-purpose machine learning.

Conclusion and Future Directions

The use of mathematical structures to "force" physical behavior—rather than merely encouraging it through loss functions—makes these models exceptionally robust, particularly when dealing with noisy or incomplete data.

Future work will focus on automating rank estimation for the dissipative matrix $\mathbf{M}$ and scaling these bracket-based GNNs to massive graphs. As machine learning continues to integrate more deeply with physical sciences, frameworks like these provide a mathematically rigorous path toward reliable and interpretable AI.

Summary updated

BibTeX
@misc{gruber2024learning,
  author = {Anthony Gruber and Kookjin Lee and Haksoo Lim and Noseong Park and Nathaniel Trask},
  title = {{Learning metriplectic systems and other bracket-based dynamics}},
  year = {2024},
  doi = {10.2172/2565147},
}
Figure