A global Monte-Carlo method for fitting parameters of differential equation models

Mariano Beguerisse-Díaz, Baojun Wang, Radhika Desikan, Mauricio Barahona

Abstract

Estimating parameters from data is a key stage of the modelling process, particularly in biological systems where many parameters need to be estimated from sparse and noisy data sets. Over the years, a variety of heuristics have been proposed to solve this complex optimisation problem, with good results in some cases yet with limitations in the biological setting. In this work, we develop an algorithm for model parameter fitting that combines ideas from evolutionary algorithms, sequential Monte Carlo and direct search optimisation. Our method performs well even when the order of magnitude and/or the range of the parameters is unknown. The method refines iteratively a sequence of parameter distributions through local optimisation combined with partial resampling from a historical prior defined over the support of all previous iterations. We exemplify our method with biological models using both simulated and real experimental data and estimate the parameters efficiently even in the absence of a priori knowledge about the parameters.

Introduction: The Challenge of Parameter Estimation

In systems biology, researchers frequently employ Ordinary Differential Equations (ODEs) to model the dynamics of molecular and cellular populations. However, the utility of these models depends entirely on the accuracy of their parameters, such as reaction rates or binding constants. Estimating these parameters is notoriously difficult due to "rugged" error landscapes filled with local minima, sparse and noisy experimental data, and the often-unknown scales of the parameters themselves.

Traditional methods like Approximate Bayesian Computation - Sequential Monte Carlo (ABC-SMC) are robust but computationally expensive and typically require the true parameter values to reside within a predefined initial search range. To address these limitations, we introduce the "Squeeze-and-Breathe" algorithm, a hybrid framework that blends global exploration with local precision.

Methods: The Squeeze-and-Breathe Framework

The Squeeze-and-Breathe algorithm operates through an iterative loop consisting of two primary phases designed to balance exploitation and exploration.

Overview of the Squeeze-and-Breathe algorithm workflow, from initial data measurements to final parameter fit.
  1. The Squeeze: The algorithm begins with a population of parameter sets. Each set is subjected to local optimization—specifically the Nelder-Mead simplex method—to find the nearest local minimum. The population is then ranked, and only the highest-performing solutions (those with the lowest error) are retained. This "squeezes" the parameter distribution toward high-quality regions of the search space.
  2. The Breathe: To prevent the algorithm from becoming trapped in local minima and to facilitate the discovery of new regions, the algorithm "breathes." It maintains a "historical prior"—a uniform distribution covering the support of all previous iterations. The next generation is created by taking a convex mixture of the current best solutions (the posterior) and this expanded historical prior.

This process continues until the error stabilizes and the parameter distributions between iterations become statistically indistinguishable, as determined by the Mann-Whitney rank sum test.

Key Results and Insights

The algorithm was validated across three distinct biological scenarios: a simulated gene-regulation model (BPM), a classic epidemiological model (SIR), and real experimental data from an inducible genetic switch.

Escaping the Prior

A significant advantage of this method is its ability to "walk" into new regions of parameter space. Unlike standard Monte Carlo methods that are restricted to their initial distributions, Squeeze-and-Breathe can identify optimal parameters even if they lie outside the initial search range. In the BPM model, the algorithm successfully identified the true parameter $\alpha = 240$ despite an initial search cap of $100$.

Convergence table for the BPM model, demonstrating the algorithm's ability to identify parameters outside the initial search range.

Handling Multi-scale Parameters and Real-world Noise

The algorithm demonstrated high efficiency in handling parameters spanning several orders of magnitude without requiring specialized scaling. In the genetic switch case study, using real-world noisy data, the method provided parameter estimates that closely matched experimental time courses across multiple induction levels. Notably, it corrected previous literature estimates that had relied on steady-state assumptions, proving that fitting full dynamical models yields more faithful biological representations.

Discussion and Limitations

While powerful, the Squeeze-and-Breathe algorithm involves a trade-off in computational cost. Because each point in the population undergoes local optimization, a single iteration can require hundreds of thousands of objective function evaluations. This may be prohibitive for computationally "stiff" models. Furthermore, as a heuristic, it does not guarantee finding the global minimum in every run; multiple independent runs are recommended to ensure consistency.

Future work will focus on incorporating stochastic ranking strategies for complex constraints and extending the framework to facilitate automated model selection.

Summary updated

Abstract

Estimating parameters from data is a key stage of the modelling process, particularly in biological systems where many parameters need to be estimated from sparse and noisy data sets. Over the years, a variety of heuristics have been proposed to solve this complex optimisation problem, with good results in some cases yet with limitations in the biological setting. Results: In this work, we develop an algorithm for model parameter fitting that combines ideas from evolutionary algorithms, sequential Monte Carlo and direct search optimisation. Our method performs well even when the order of magnitude and/or the range of the parameters is unknown. The method refines iteratively a sequence of parameter distributions through local optimisation combined with partial resampling from a historical prior defined over the support of all previous iterations. We exemplify our method with biological models using both simulated and real experimental data and estimate the parameters efficiently even in the absence of a priori knowledge about the parameters.
BibTeX
@article{beguerissediaz2012global,
  author = {Mariano Beguerisse-Díaz and Baojun Wang and Radhika Desikan and Mauricio Barahona},
  title = {{A global Monte-Carlo method for fitting parameters of differential equation models}},
  journal = {Journal of The Royal Society Interface},
  year = {2012},
  doi = {10.1098/rsif.2011.0767},
}
Figure