Abstract
Parameter estimation for nonlinear dynamic system models, represented by ordinary differential equations (ODEs), using noisy and sparse experimental data is a vital task in many fields. We propose a fast and accurate method, MAGI (MAnifold-constrained Gaussian process Inference), for this task. MAGI uses a Gaussian process model over time-series data, explicitly conditioned on the constraint that derivatives of the Gaussian process must satisfy the ODE system. By doing so, we completely bypass the need for numerical integration and achieve substantial savings in computational time. MAGI is also suitable for inference with unobserved system components, which often occur in real experiments. MAGI is distinct from existing approaches as we provide a principled statistical construction under a Bayesian framework. We demonstrate the accuracy and speed of MAGI using realistic examples based on physical experiments.
Introduction: Bypassing the Numerical Integration Bottleneck
Ordinary Differential Equations (ODEs) are fundamental to scientific modeling, describing complex dynamics in fields ranging from gene regulation to epidemiology. However, estimating parameters for these models from real-world data is challenging due to noise, sparsity, and the frequent presence of unobserved variables. The traditional "Gold Standard" for parameter estimation relies on numerical integration (e.g., Runge-Kutta methods), requiring thousands of ODE solutions during optimization or sampling. This creates a significant computational bottleneck.
While "smoothing" methods using B-splines or standard Gaussian Processes (GPs) attempt to skip integration, they often lack a rigorous statistical foundation or fail to handle unobserved variables effectively. We introduce MAGI (MAnifold-constrained Gaussian process Inference), a novel Bayesian framework that addresses these limitations by treating system trajectories as realizations of a GP constrained to satisfy the underlying ODEs.
Methodology: Manifold-Constrained Inference
The core innovation of MAGI is the manifold constraint. Instead of simply fitting a GP to data, we explicitly condition the GP such that its derivatives satisfy the ODE system: $\dot{X}(t) = f(X(t), \theta, t)$. This approach transforms a global optimization problem into a local derivative-matching problem, entirely bypassing the need for numerical integration.
Key Components
- Bayesian Framework: We construct a joint posterior distribution for model parameters ($\theta$) and system trajectories ($X$).
- Finite Discretization: The continuous manifold constraint is approximated using a finite set of discretization points, making the problem computationally tractable.
- Hamiltonian Monte Carlo (HMC): To efficiently sample from the high-dimensional posterior, we employ HMC, which outperforms standard sampling methods in this context.
- Differentiable Kernels: We utilize Matern kernels (specifically $v=2.01$) to ensure the GP is sufficiently differentiable for the derivative-based constraints to hold.
Experimental Evaluation
MAGI was evaluated on three benchmark dynamic systems:
- Hes1 Model: A gene regulation system where one component ($H$) is entirely unobserved, and others are sampled asynchronously.
- FitzHugh-Nagumo (FN) Model: A standard model for neuronal spike potentials.
- Protein Transduction Model: A complex 5-component model with non-identifiable parameters, serving as a stress test for trajectory recovery.
Key Results
MAGI demonstrates significant improvements in both speed and accuracy compared to state-of-the-art methods. On the FitzHugh-Nagumo model, MAGI achieved a 145x speedup over FGPGM and a 90x speedup over AGM.
Beyond speed, MAGI excels in accuracy, particularly for systems with hidden components. In the Hes1 model, MAGI successfully reconstructed the trajectory of the unobserved $H$ component by "borrowing" information from observed variables through the ODE-linked manifold constraint.
As shown in the figure above, even with noisy and sparse observations (leftmost panel), MAGI's inferred trajectories (green) closely track the ground truth (red) for both observed (P, M) and unobserved (H) components, with well-calibrated 95% uncertainty intervals (blue).
Conclusion and Future Directions
MAGI provides a principled, integration-free approach to ODE inference. By leveraging manifold constraints within a Bayesian GP framework, it achieves order-of-magnitude speedups while maintaining high accuracy, even in the presence of unobserved variables. Future work includes extending this framework to Stochastic Differential Equations (SDEs) and exploring potential integrations with deep learning architectures such as Neural Tangent Kernels.
Summary updated