Abstract
The behavior of many dynamical systems follow complex, yet still unknown partial differential equations (PDEs). While several machine learning methods have been proposed to learn PDEs directly from data, previous methods are limited to discrete-time approximations or make the limiting assumption of the observations arriving at regular grids. We propose a general continuous-time differential model for dynamical systems whose governing equations are parameterized by message passing graph neural networks. The model admits arbitrary space and time discretizations, which removes constraints on the locations of observation points and time intervals between the observations. The model is trained with continuous-time adjoint method enabling efficient neural PDE inference. We demonstrate the model's ability to work with unstructured grids, arbitrary time steps, and noisy observations. We compare our method with existing approaches on several well-known physical systems that involve first and higher-order PDEs with state-of-the-art predictive performance.
Introduction
Partial Differential Equations (PDEs) are fundamental to the natural sciences, describing phenomena from fluid dynamics to atmospheric patterns. Traditionally, discovering these equations required significant domain expertise. Recent machine learning approaches have attempted to "learn" PDEs from data, but most are constrained by grid dependency (assuming regular, pixel-like grids) and discrete-time assumptions (assuming fixed observation intervals).
This paper addresses these limitations by proposing a "continuous-time, free-form" learner capable of handling the sparse, irregularly sampled data typical of real-world physical monitoring.
Methodology
The proposed framework combines three powerful concepts to enable flexible PDE learning:
- The Method of Lines (MOL): Instead of simultaneous discretization, MOL discretizes only the spatial domain into $N$ nodes. This transforms a single PDE into a system of $N$ coupled Ordinary Differential Equations (ODEs).
- MPNNs as the Differential Function: The core physics—how a point's state changes based on its neighbors—is learned by a Message Passing Neural Network. To define "neighbors" in unstructured space, the authors utilize Delaunay triangulation. The MPNN is position-invariant, focusing on local physical rules rather than specific coordinates.
- Continuous-Time Adjoint Method: To ensure efficient training, the model uses the adjoint method popularized in Neural ODE literature. This allows for gradient computation without storing intermediate ODE solver states, keeping memory costs independent of the number of time steps.
Experiments and Results
The model was evaluated on synthetic data from three classic systems: the Convection-Diffusion Equation, the Heat Equation, and Burgers’ Equations. Data was downsampled to test performance across varying grid densities (750 to 3000 nodes) and sparse temporal intervals.
Key Insights
- Continuous Time Advantage: By learning the system's derivative, the model can be queried at any arbitrary time point. This makes it robust to missing data; the model can recover dynamics even when trained on extremely sparse time points.
- Relative Geometry: A critical finding was that including relative positional information ($x_j - x_i$) in graph edges is essential. This allows the model to capture directional physics, such as convection, where neighbor orientation dictates flow.
- Zero-Shot Grid Generalization: Models trained on coarse grids (750 nodes) generalize effectively to finer grids (3000 nodes), suggesting the MPNN captures underlying local physical operators rather than just memorizing the training grid.
Performance
The proposed model achieved state-of-the-art predictive performance compared to baselines like PDE-Net and DPGN. It demonstrated high temporal robustness, maintaining stable error rates as observation intervals increased. Additionally, the framework proved robust to additive noise and showed clear performance scaling with increased training data.
Limitations and Future Work
Despite its strengths, the model's reliance on ODE solvers makes it more computationally expensive than discrete-time CNN-based approaches. Furthermore, the current assumption of local interactions may struggle with systems involving long-range forces. Future research will focus on extending the model to handle latent variables in partially observed systems.
Summary updated