Distribution approximations for the chemical master equation: comparison of the method of moments and the system size expansion

Alexander Andreychenko, Luca Bortolussi, Ramon Grima, Philipp Thomas, Verena Wolf

This paper provides a rigorous comparison of two leading mathematical frameworks used to understand the "noise" in biochemical systems: the Method of Moments (MM) and the System Size Expansion (SSE).

TL;DR

Researchers compared two different ways to approximate the probability distributions of stochastic chemical reactions without relying on slow, expensive simulations. They found that while both the Method of Moments and System Size Expansion can accurately capture complex behaviors like multimodality, they offer distinct trade-offs between numerical stability and analytical simplicity.

What Problem Does This Paper Address?

In the microscopic world of a living cell, chemical reactions are inherently "noisy." Because molecule counts are often low, reactions happen at random intervals, leading to significant fluctuations in population sizes. To model this, scientists use the Chemical Master Equation (CME).

The problem? The CME is notoriously difficult to solve. For most realistic biological networks, the number of possible states is so massive that a direct numerical solution is impossible. While the Stochastic Simulation Algorithm (SSA) provides a "gold standard" via Monte Carlo methods, it is computationally expensive and slow. This paper explores faster, approximate methods that can provide the entire probability distribution of a system—essential for understanding phenomena like gene "switching" or cell-to-cell variability—without the overhead of thousands of simulations.

Methods

The authors pit two approximation strategies against each other:

  1. Method of Moments (MM) + Maximum Entropy:
    • Instead of tracking every possible state, this method tracks statistical moments (mean, variance, skewness, etc.).
  2. Because equations for lower moments often depend on higher ones (the "closure problem"), the authors use a low-dispersion closure.
  3. To turn these abstract numbers back into a visual distribution, they apply the Maximum Entropy Principle, which finds the most "unbiased" distribution that fits the calculated moments.
  4. System Size Expansion (SSE):
    • This is an analytical approach that treats the system as a deterministic "average" plus a fluctuating component.
  5. While many researchers stop at the Linear Noise Approximation (LNA)—which assumes fluctuations are Gaussian—this paper goes further. It includes higher-order terms to capture non-Gaussian features.
  6. A key refinement here is performing the expansion about the true mean rather than the deterministic limit, which significantly improves accuracy for small populations.

For complex systems with multiple peaks (multimodality), the authors utilize hybrid/conditional versions of both methods, which treat small, switching populations (like a gene turning on/off) differently than large, fluctuating populations (like protein counts).

Datasets

The paper evaluates these methods using two classic benchmark models in synthetic biology:

  • Example 1: Bursty Protein Production. A model where mRNA is produced in bursts and proteins are degraded by enzymes (Michaelis-Menten kinetics). This produces a skewed, unimodal distribution.
  • Example 2: Self-Activating Gene. A model where a protein promotes its own production, creating a "switch" that leads to a multimodal (multi-peaked) distribution.

Ground truth for both was established using the Stochastic Simulation Algorithm (SSA).

Key Insights and Ideas

  • Beyond the Gaussian: The standard Linear Noise Approximation (LNA) often fails in biology because it assumes everything looks like a Bell curve. This paper demonstrates that adding just a few higher-order terms in the SSE or a few extra moments in the MM can capture the "skew" and "heavy tails" seen in real cells.
  • The "Stiffness" Wall: The Method of Moments is highly flexible, but it hits a numerical ceiling. As you try to include more moments (e.g., beyond the 7th or 8th) to increase accuracy, the underlying differential equations become "stiff," causing numerical solvers to fail.
  • The Power of Conditioning: For genes that switch slowly between states, "conditional" approximations are essential. By calculating distributions given a specific gene state and then summing them, both methods can accurately predict multimodality.
  • Mean Matters: For the SSE, expanding around the actual stochastic mean (rather than the deterministic rate equation solution) is crucial for preventing "undulations" or negative probabilities in the results.

Key Results

  • Accuracy: In the bursty protein case, the SSE (truncated at $\Omega^{-3}$) achieved a percentage error of 2.8%, while the MM (using 5 moments) achieved 2.0%. Both are remarkably close to the "exact" SSA results.
  • Multimodality: In the self-activating gene model, the conditional SSE significantly outperformed the standard Gaussian approximation, which correctly identified the existence of two peaks but missed their precise locations.
  • Computational Efficiency: Both methods are orders of magnitude faster than running the thousands of SSA trajectories required to build a smooth histogram.

Limitations and Future Work

  • Dimensionality: The Maximum Entropy reconstruction becomes numerically unstable and "ill-conditioned" as the number of chemical species (dimensions) increases.
  • Species Limits: The specific analytical SSE solutions provided in the paper are currently limited to single-species models or those with clear timescale separations.
  • Boundary Issues: Both methods still struggle slightly at the "zero-molecule" boundary, where the discrete nature of the system is most pronounced.
  • Scaling: Future work is needed to scale these approximations to large-scale metabolic or signaling networks without losing the ability to capture non-Gaussian noise.

Summary updated

Abstract

The stochastic nature of chemical reactions involving randomly fluctuating population sizes has lead to a growing research interest in discrete-state stochastic models and their analysis. A widely-used approach is the description of the temporal evolution of the system in terms of a chemical master equation (CME). In this paper we study two approaches for approximating the underlying probability distributions of the CME. The first approach is based on an integration of the statistical moments and the reconstruction of the distribution based on the maximum entropy principle. The second approach relies on an analytical approximation of the probability distribution of the CME using the system size expansion, considering higher-order terms than the linear noise approximation. We consider gene expression networks with unimodal and multimodal protein distributions to compare the accuracy of the two approaches. We find that both methods provide accurate approximations to the distributions of the CME while having different benefits and limitations in applications.
BibTeX
@misc{andreychenko2015distribution,
  author = {Alexander Andreychenko and Luca Bortolussi and Ramon Grima and Philipp Thomas and Verena Wolf},
  title = {{Distribution approximations for the chemical master equation: comparison of the method of moments and the system size expansion}},
  year = {2015},
  eprint = {1509.09104},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1509.09104},
}
Figure