Systematic comparison of modeling fidelity levels and parameter inference settings applied to negative feedback gene regulation

Adrien Coulier, Prashant Singh, Marc Sturrock, Andreas Hellander

Introduction

In mathematical biology, researchers face a persistent trade-off between model fidelity and computational cost. While "well-mixed" models are computationally efficient, they often overlook critical spatial effects like diffusion. Conversely, high-fidelity spatial stochastic models offer realism at the expense of significant computational time. This study, published in PLOS Computational Biology, addresses the challenge of choosing the appropriate level of model complexity for parameter inference. By developing a high-performance pipeline that leverages machine learning, the authors demonstrate that increased model detail is often necessary for accuracy, even when experimental data appears simple.

Methods

The study centers on the Hes1 negative feedback system, a fundamental gene regulatory motif characterized by oscillatory behavior. The authors compared three distinct modeling fidelity levels:

  1. Smoldyn: A high-fidelity, particle-based spatial stochastic model.
  2. Compartment-Based Model (CBM): A mid-fidelity multiscale approach dividing the cell into nuclear and cytoplasmic compartments.
  3. Well-Mixed Model (WMM): A standard approach utilizing the Stochastic Simulation Algorithm (SSA).
Schematic of the Hes1 negative feedback system used as the benchmark model.

To facilitate the extensive simulations required for parameter inference, the researchers developed a pipeline using Approximate Bayesian Computation (ABC) accelerated by Gaussian Process (GP) surrogates. Instead of executing expensive simulators for every parameter check, a GP was trained to act as a surrogate for the distance function, enabling a rapid scan of the parameter space (diffusion vs. reactivity).

Overview of the GP-accelerated parameter inference pipeline.

The pipeline was tested using synthetic "ground truth" data generated by the Smoldyn simulator across 256 parameter combinations. This allowed for a systematic evaluation of how model choice, summary statistics, and data types (mRNA vs. protein) influence inference accuracy.

Key Insights

Spatial Detail and Non-Spatial Data

A significant finding is that spatial models yield superior inference results even when the input data is non-spatial (e.g., total cellular protein counts). Spatial models inherently account for diffusion time, whereas simpler models often erroneously compensate for these delays by adjusting reaction rates, leading to biased parameter estimates.

The Model Error Ceiling

For lower-fidelity models like the WMM and CBM, the study identified a "model error ceiling." Beyond a certain point, increasing the volume of data (more time points or cells) fails to improve accuracy because the error is dominated by the model's structural inadequacies rather than statistical uncertainty.

Informational Value of Data Types

In the Hes1 system, measuring protein levels proved significantly more informative than measuring mRNA. While measuring both provided a marginal improvement, protein data alone allowed for much tighter parameter estimation. Furthermore, comparing full molecular count distributions using the Kolmogorov distance was found to be more robust than using traditional summary statistics like mean or variance.

Results and Performance

The integration of Gaussian Process surrogates achieved a massive computational speedup, reducing inference time from approximately 200 core-hours to less than one core-hour per setup. The combination of the high-fidelity Smoldyn model and Kolmogorov distance consistently outperformed other configurations. Notably, the high-fidelity model maintained its ability to discriminate between parameters even with coarse time sampling, mimicking the limitations of experimental techniques like Flow Cytometry.

Conclusion

This research provides a rigorous framework for evaluating modeling choices in systems biology. While high-fidelity models are more complex, the authors show they are often essential for avoiding biased results. Future work may involve extending this pipeline to automatically learn optimal summary statistics via neural networks and applying these techniques to multi-cellular systems.

Summary updated

Abstract

Author summary Computational models plays a vital role in modern biology and are commonly used to compare theory with data. These models can take different forms, and there is often a trade-off between model detail and the computational resources needed to simulate them. Furthermore a choice must be made regarding how to compare the model output with the available data with several different distance metrics available. The choice of model and distance metric may also be impacted by the amount of available data. Therefore deciding how best to infer model parameters from available experimental data is a challenging problem. In this paper we have developed a computational pipeline designed to systematically evaluate inference accuracy for a wide range of true known parameters. To demonstrate its use, we applied it to a well studied gene regulation model. In particular, we compared a simple model, mid-complexity model and a complex model for several data-scenarios and for multiple numerical options for parameter inference. We believe this pipeline can be used as a preliminary step to guide scientists prior to gathering experimental data. This could prevent experimentalists from gathering unnecessary expensive experimental data or modelers from expending huge computational resources on simulating superfluously complex models.
BibTeX
@article{coulier2022systematic,
  author = {Adrien Coulier and Prashant Singh and Marc Sturrock and Andreas Hellander},
  title = {{Systematic comparison of modeling fidelity levels and parameter inference settings applied to negative feedback gene regulation}},
  journal = {PLOS Computational Biology},
  year = {2022},
  doi = {10.1371/journal.pcbi.1010683},
}
Figure