Exact and efficient hybrid Monte Carlo algorithm for accelerated Bayesian inference of gene expression models from snapshots of single-cell transcripts

Yen Ting Lin, Nicolas E. BuchlerThe Journal of Chemical Physics 2019 · DOI

TL;DR

This paper introduces a novel hybrid Monte Carlo algorithm (PM-PDMSR) that significantly accelerates the simulation of gene expression models, enabling efficient Bayesian inference for complex models from single-cell data. This advancement allows for more robust model selection and parameter uncertainty quantification, particularly crucial for small datasets common in single-cell experiments.

What Problem Does This Paper Address?

Understanding how genes are expressed within individual cells is fundamental to biology. This process is inherently noisy and stochastic, leading to significant cell-to-cell variability in transcript levels. To decipher the underlying regulatory mechanisms, scientists need to build and test mathematical models of gene expression. Bayesian inference is the gold standard for this, offering robust parameter estimation and model selection, especially with limited data. However, existing computational methods for simulating these models are prohibitively slow, making Bayesian inference impractical for typical gene expression models. This paper tackles this computational bottleneck, aiming to make sophisticated Bayesian analysis of gene expression feasible.

Methods

The core innovation is a hybrid simulation method called the Poisson Mixture with a Piecewise Deterministic Markov Switching Rate (PM-PDMSR). The authors first derive a key theoretical result: the time-dependent mRNA distribution of discrete-state gene expression models can be viewed as a dynamic Poisson mixture. The "mixing kernel" of this mixture is governed by a piecewise deterministic Markov process (PDMP), which describes the transitions between different gene states (e.g., active vs. inactive).

The PM-PDMSR algorithm leverages this insight:

  1. Kinetic Monte Carlo (KMC) sampling: It efficiently generates sample paths of the underlying gene state transitions (the PDMP).
  2. Analytical solution for mRNA distribution: For each gene state trajectory, the mRNA distribution is calculated analytically as a Poisson distribution with a time-dependent rate.
  3. Convolution: The final joint distribution of gene states and mRNA levels is obtained by averaging (convolving) the Poisson distributions from all sampled gene state trajectories.

This hybrid approach is significantly faster than traditional methods that rely on numerically solving the Chemical Master Equation (CME), which involves solving a large system of differential equations. The PM-PDMSR method is then integrated into an existing Bayesian inference framework (BayFISH) to perform parameter estimation and model selection.

Datasets

The authors used synthetic datasets generated from known two-allele, three-state gene expression models. These datasets were created to mimic typical single-cell RNA sequencing or smFISH data, with varying numbers of cells sampled at different time points after an induction event. These synthetic datasets serve as benchmarks to evaluate the accuracy and efficiency of their proposed method and compare it against existing approaches. No new experimental datasets were introduced.

Key Insights and Ideas

  • Gene expression dynamics as dynamic Poisson mixtures: This is a fundamental theoretical insight that bridges the gap between discrete gene state models and continuous mRNA distributions. It provides a powerful analytical framework for understanding the stochasticity.
  • PM-PDMSR offers a 1000x speedup: The hybrid simulation method is orders of magnitude faster than traditional CME solvers, making complex Bayesian analyses computationally tractable.
  • Bayesian evidence is superior for model selection: The paper strongly advocates for using Bayesian evidence (marginalized likelihood) over approximate metrics like BIC and AIC, especially for small datasets, as it more rigorously quantifies model likelihood and avoids overfitting.
  • Parameter inference is robust even with modest data: For a known model, kinetic parameters can be reasonably constrained even with relatively small sample sizes (e.g., 100 cells per time point).
  • "Sloppy" models are common: The paper acknowledges the phenomenon of "sloppiness" in biological models, where multiple parameters can be perturbed simultaneously without significantly affecting the model's fit to data. This highlights the importance of full posterior distributions for uncertainty quantification.

Key Results

  • The PM-PDMSR algorithm accelerates the calculation of time-dependent mRNA distributions by up to 1000-fold compared to standard CME integration methods.
  • The hybrid method demonstrates comparable accuracy to CME simulations.
  • The PM-PDMSR simulator exhibits efficient parallelization on multi-core processors, scaling well with the number of cores. In contrast, CME solvers showed poor scaling.
  • Bayesian inference using the accelerated method successfully constrained kinetic parameters for synthetic datasets, with uncertainty decreasing significantly as sample size increased.
  • Bayesian evidence correctly identified the true model among many competing candidates, outperforming BIC and AIC, particularly with smaller datasets.
  • For model selection, a 10-fold increase in data sampling (from 100 to 1000 cells per time point) was shown to be crucial for reliably distinguishing between models.

Limitations and Future Work

  • Computational cost for model selection: While significantly accelerated, model selection still involves evaluating many candidate models, which can be computationally intensive. The authors suggest that for very large datasets where posteriors approach Gaussian distributions, simpler methods like BIC might suffice, but their method remains more efficient for estimating the maximum likelihood.
  • "Sloppy" parameter spaces: The paper notes that posterior distributions can deviate from Gaussian limits, requiring consideration of the full posterior for accurate uncertainty quantification.
  • Future work directions:
    • Investigating more efficient MCMC samplers like Hamiltonian Monte Carlo (HMC) for parameter inference.
  • Developing methods to accelerate the convolution step in PM-PDMSR.
  • Exploring likelihood-free inference methods to bypass explicit likelihood calculations.
  • Using Bayesian evidence and posterior distributions to guide the design of future experiments (e.g., optimal sampling times and densities).

Summary updated

Abstract

Single cells exhibit a significant amount of variability in transcript levels, which arises from slow, stochastic transitions between gene expression states. Elucidating the nature of these states and understanding how transition rates are affected by different regulatory mechanisms require state-of-the-art methods to infer underlying models of gene expression from single cell data. A Bayesian approach to statistical inference is the most suitable method for model selection and uncertainty quantification of kinetic parameters using small data sets. However, this approach is impractical because current algorithms are too slow to handle typical models of gene expression. To solve this problem, we first show that time-dependent mRNA distributions of discrete-state models of gene expression are dynamic Poisson mixtures, whose mixing kernels are characterized by a piecewise deterministic Markov process. We combined this analytical result with a kinetic Monte Carlo algorithm to create a hybrid numerical method that accelerates the calculation of time-dependent mRNA distributions by 1000-fold compared to current methods. We then integrated the hybrid algorithm into an existing Monte Carlo sampler to estimate the Bayesian posterior distribution of many different, competing models in a reasonable amount of time. We demonstrate that kinetic parameters can be reasonably constrained for modestly sampled data sets if the model is known a priori. If there are many competing models, Bayesian evidence can rigorously quantify the likelihood of a model relative to other models from the data. We demonstrate that Bayesian evidence selects the true model and outperforms approximate metrics typically used for model selection.
BibTeX
@article{lin2019exact,
  author = {Yen Ting Lin and Nicolas E. Buchler},
  title = {{Exact and efficient hybrid Monte Carlo algorithm for accelerated Bayesian inference of gene expression models from snapshots of single-cell transcripts}},
  journal = {The Journal of Chemical Physics},
  year = {2019},
  doi = {10.1063/1.5110503},
}
Figure