Enhancing the Inductive Biases of Graph Neural ODE for Modeling Dynamical Systems

Suresh Bishnoi, Ravinder Bhattoo, Sayan Ranu, N. M. Anoop Krishnan

TL;DR

This paper introduces MCGNODE, a graph-based Neural ODE that incorporates physical inductive biases like explicit constraints and Newton’s third law. By combining the topology-awareness of Graph Neural Networks with these priors, the model achieves zero-shot generalization to unseen system sizes and outperforms standard energy-conserving models (like Hamiltonian Graph Networks) by up to four orders of magnitude in energy conservation.

What Problem Does This Paper Address?

Modeling the dynamics of physical systems—like a swinging pendulum or a vibrating spring—is a cornerstone of science and engineering. While deep learning has made strides here, two major hurdles remain:

  1. Generalization (Transductivity): Most existing models, including standard Neural ODEs (NODEs) and Hamiltonian Neural Networks (HNNs), are "transductive." If you train them on a 3-link pendulum, they can't simulate a 5-link one. They don't understand the underlying "rules" of the parts; they only learn the specific system they were shown.
  2. Physical Consistency: Standard neural networks often "drift," violating fundamental laws like the conservation of energy or momentum over long simulations.

The authors aim to create a model that is both inductive (can generalize to any system size) and physically grounded (respects conservation laws and constraints).

Methods

The authors propose GNODE (Graph Neural ODE) and iteratively enhance it with three key physical priors:

  • Topology-Awareness (GNODE): By using a Graph Neural Network (GNN) to represent particles as nodes and interactions as edges, the model learns local interaction rules rather than global system states. This enables it to generalize to systems with more or fewer particles than seen during training.
  • Decoupling Dynamics and Constraints (CGNODE): Instead of learning a single "black box" acceleration, the model explicitly separates the forces from the system's geometric constraints (like the fixed length of a pendulum rod) using D’Alembert’s principle.
  • Decoupling Internal and Body Forces (CDGNODE): The architecture is split to handle local internal forces (e.g., spring tension) and global body forces (e.g., gravity) separately. This makes the model more efficient and interpretable.
  • Newton’s Third Law (MCGNODE): The "Momentum Conserving" variant enforces that the force particle A exerts on B is equal and opposite to the force B exerts on A ($f_{ij} = -f_{ji}$). The authors theoretically prove this ensures exact linear momentum conservation.
Figure 1: Architecture of CDGnode (see App. K).

For integration, they use the velocity Verlet algorithm, a symplectic integrator that is inherently better at conserving energy over long periods than standard methods like Runge-Kutta 4.

Datasets

The models were evaluated on synthetic datasets generated from classical mechanics:

  • n-pendulum systems: Chains of 3, 4, and 5 links.
  • n-spring systems: Chains of 3, 4, and 5 links.
  • Generalization tests: Models trained on 5-link systems were tested on 10- and 20-link pendulums, and 20- and 50-link spring systems.
  • 3D Solid: A complex $5 \times 5 \times 5$ cube (125 particles) simulated using peridynamics to test large-scale applicability.

Key Insights and Ideas

  • Inductive Bias > Model Complexity: You don't need a more complex "black box" to get better results; you need a smarter one. Adding Newton's third law as a structural constraint (MCGNODE) was the "aha moment" that stabilized energy violation errors that otherwise diverged in simpler models.
  • Symplectic Integration is a Free Lunch: Using the Verlet integrator allows the model to stay "on the manifold" of energy conservation without needing the complex and computationally expensive double-differentiation required by Lagrangian or Hamiltonian Neural Networks.
  • Zero-Shot is Possible: Because the model learns at the node and edge level, a GNODE trained on a small system can simulate a much larger one immediately, without further training.

Key Results

  • Superior Accuracy: MCGNODE outperformed the graph versions of Lagrangian (LGN) and Hamiltonian (HGN) networks. It reduced energy violation errors by ~4 orders of magnitude for pendulums and ~2 orders for spring systems.
  • Momentum Conservation: As shown in Figure 4, MCGNODE achieves near-zero momentum error, whereas other models (including HGN) see their errors explode over time.
Comparison of momentum error across different models.
  • Robust Generalization: The model successfully simulated a 50-link spring system after being trained only on a 5-link system, maintaining significantly lower rollout errors than baselines.
  • Computational Efficiency: MCGNODE is significantly faster than LGN and HGN (up to 20x faster in some spring system cases) because it predicts forces directly rather than differentiating a learned energy scalar.

Limitations and Future Work

While powerful, the authors note several areas for growth:

  • Simple Interactions: The current work focuses on particle-based systems. Real-world dynamics involving rigid body contacts, friction, and collisions are not yet addressed.
  • Material Complexity: The model doesn't currently account for plastically deformable or viscoelastic materials where the physics depends on the rate of strain.
  • Constraint Learning: Currently, constraints (like rod lengths) are provided to the model. A promising future direction is learning these constraints directly from raw trajectory data.

Summary updated

Abstract

Neural networks with physics based inductive biases such as Lagrangian neural networks (LNN), and Hamiltonian neural networks (HNN) learn the dynamics of physical systems by encoding strong inductive biases. Alternatively, Neural ODEs with appropriate inductive biases have also been shown to give similar performances. However, these models, when applied to particle based systems, are transductive in nature and hence, do not generalize to large system sizes. In this paper, we present a graph based neural ODE, GNODE, to learn the time evolution of dynamical systems. Further, we carefully analyse the role of different inductive biases on the performance of GNODE. We show that, similar to LNN and HNN, encoding the constraints explicitly can significantly improve the training efficiency and performance of GNODE significantly. Our experiments also assess the value of additional inductive biases, such as Newtons third law, on the final performance of the model. We demonstrate that inducing these biases can enhance the performance of model by orders of magnitude in terms of both energy violation and rollout error. Interestingly, we observe that the GNODE trained with the most effective inductive biases, namely MCGNODE, outperforms the graph versions of LNN and HNN, namely, Lagrangian graph networks (LGN) and Hamiltonian graph networks (HGN) in terms of energy violation error by approx 4 orders of magnitude for a pendulum system, and approx 2 orders of magnitude for spring systems. These results suggest that competitive performances with energy conserving neural networks can be obtained for NODE based systems by inducing appropriate inductive biases.
BibTeX
@inproceedings{bishnoi2023enhancing,
  author = {Suresh Bishnoi and Ravinder Bhattoo and Sayan Ranu and N. M. Anoop Krishnan},
  title = {{Enhancing the Inductive Biases of Graph Neural ODE for Modeling Dynamical Systems}},
  booktitle = {International Conference on Learning Representations},
  year = {2023},
  url = {https://openreview.net/forum?id=ATLEl_izD87},
}
Figure