Abstract
Often the underlying system of differential equations driving a stochastic dynamical system is assumed to be known, with inference conditioned on this assumption. We present a Bayesian framework for discovering this system of differential equations under assumptions that align with real-life scenarios, including the availability of relatively sparse data. Further, we discuss computational strategies that are critical in teasing out the important details about the dynamical system and algorithmic innovations to solve for acute parameter interdependence in the absence of rich data. This gives a complete Bayesian pathway for model identification via a variable selection paradigm and parameter estimation of the corresponding model using only the observed data.
Introduction
The discovery of governing laws from data is a cornerstone of modern physics and engineering. While traditional inference focuses on parameter estimation for a fixed model structure, real-world systems—such as climate patterns or molecular dynamics—often lack a known "drift function." The challenge of identifying these equations is compounded by noisy observations, latent variables, and the acute parameter interdependence characteristic of stochastic systems. A recent paper by Gupta, Vats, and Chatterjee provides a robust Bayesian roadmap for discovering the mathematical structure of a system from scratch, even when data is sparse and the system is chaotic.
Methodology
Equation Selection via Spike-and-Slab Priors
To identify the correct model structure, the researchers utilize a "dictionary" of potential functional terms (e.g., linear, quadratic, cross-products). They employ spike-and-slab priors to perform variable selection. Each term in the dictionary is associated with a binary indicator variable ($\gamma$):
- The Slab: When $\gamma=1$, the term is included, and its coefficient is drawn from a diffuse prior.
- The Spike: When $\gamma=0$, the coefficient is constrained to a narrow region around zero, effectively removing the term from the model.
The Linchpin Sampler
A significant hurdle in stochastic system inference is the "vulnerable" dependence between the estimated latent path ($\mathbf{X}$) and the diffusion covariance ($\Sigma$). Standard MCMC methods often struggle with slow convergence due to this coupling; a tiny change in one causes a "butterfly effect" in the other. The authors implement a linchpin variable MCMC sampler that marginalizes out $\Sigma$. By integrating this component out of the posterior, they break the dependency, allowing for more efficient exploration of the parameter space and significantly higher effective sample sizes.
Experimental Results
The framework was validated using three benchmark dynamical systems: the Ornstein-Uhlenbeck (OU) process, the Lorenz-63 butterfly attractor, and the Lorenz-96 atmospheric model. These systems were simulated to represent sparse, real-world conditions, using only 20 to 400 noisy observations over short time intervals.
Correct Identification and Accuracy
Despite the sparsity of the data, the model consistently identified the true underlying terms while "switching off" irrelevant ones. In the Lorenz-63 system, for instance, the drift parameters were estimated with high precision (e.g., a true value of 10.0 was estimated as 9.9).
Robustness and Efficiency
The "linchpin" sampler demonstrated superior mixing properties, with effective sample sizes nearly five times higher than traditional Metropolis-Hastings approaches. Furthermore, the system proved robust to initialization; the correct structure was identified even when the initial latent path was a simple linear interpolation of noisy data points.
Conclusion
This Bayesian pathway provides a complete solution for model identification and parameter estimation in stochastic dynamical systems. By addressing the dual challenges of model uncertainty and computational gridlock, this work moves toward "automated science," where mathematical laws can be extracted directly from sparse, real-world observations. Future work will focus on estimating unknown measurement noise and reducing discretization errors through advanced functional representations such as splines or basis functions.
Summary updated