Abstract
Neural networks used for multi-interaction trajectory reconstruction lack the ability to estimate the uncertainty in their outputs, which would be useful to better analyse and understand the systems they model. In this paper we extend the Factorised Neural Relational Inference model to output both a mean and a standard deviation for each component of the phase space vector, which together with an appropriate loss function, can account for uncertainty. A variety of loss functions are investigated including ideas from convexification and a Bayesian treatment of the problem. We show that the physical meaning of the variables is important when considering the uncertainty and demonstrate the existence of pathological local minima that are difficult to avoid during training.
Introduction
Deep learning has become remarkably effective at learning physics—predicting the trajectories of interacting particles and even inferring underlying forces. However, most current models provide only point estimates, lacking the quantified uncertainty essential for scientific and engineering applications. Uncertainty quantification helps distinguish between computational errors (e.g., finite time-steps), physical errors (e.g., chaotic dynamics), and model-induced errors.
This study utilizes a simulated dataset of five point particles in a 2D box, interacting via spring and charge forces. The simulations include elastic wall collisions and are integrated using a leapfrog method, providing a controlled environment to study the interplay between physics and uncertainty.
Methodology
The researchers extended the Factorised Neural Relational Inference (fNRI) framework, a Variational Autoencoder (VAE) designed to infer interaction graphs and reconstruct trajectories. The primary modification involves the decoder, which is tasked with outputting both a mean ($\mu$) and a standard deviation ($\sigma$) for every component of the phase space vector (position and velocity).
The investigation explored various levels of uncertainty isotropy:
- Isotropic: A single $\sigma$ for all coordinates.
- Semi-isotropic: Separate $\sigma$ values for position and velocity.
- Anisotropic: Unique $\sigma$ for every coordinate.
Given the heavy-tailed nature of error distributions in these systems, a Lorentzian (Cauchy) loss function was implemented as an alternative to the standard Gaussian. Additionally, Bayesian priors for $\sigma$ were introduced via a Kullback-Leibler (KL) term, and convexification techniques were applied to smooth the loss landscape.
Results and Discussion
A significant finding was the importance of physical semantics. Isotropic models, which force a single uncertainty value for both position and velocity, failed because these variables have different dimensions and error scales. Semi-isotropic or anisotropic models proved necessary for accurate modeling.
The study identified a "Two Minima" trap in the loss landscape. Models often converged to one of two pathological states: either ignoring uncertainty to focus solely on reconstruction (fixed small variance) or overestimating $\sigma$ to "explain away" reconstruction errors, effectively failing to learn the underlying physics.
The Lorentzian loss function outperformed the baseline in graph inference accuracy (93.2% vs 91.5%), although trajectory reconstruction was slightly less precise. Conversely, standard Gaussian models struggled with a 29.6% accuracy unless convexification was employed to escape local minima.
Conclusion
While extending fNRI to output uncertainty is feasible, training remains brittle. The existence of pathological local minima where the model overestimates uncertainty to minimize loss poses a significant challenge. Future work will focus on refining Bayesian priors, implementing two-stage training regimes, and testing on complex real-world datasets where ground truth physics is unknown.
Summary updated