What Can We Learn from Global Sensitivity Analysis of Biochemical Systems?

Edward Kent, Stefan Neumann, Ursula Kummer, Pedro MendesPLoS 2013 · DOI

Abstract

Most biological models of intermediate size, and probably all large models, need to cope with the fact that many of their parameter values are unknown. In addition, it may not be possible to identify these values unambiguously on the basis of experimental data. This poses the question how reliable predictions made using such models are. Sensitivity analysis is commonly used to measure the impact of each model parameter on its variables. However, the results of such analyses can be dependent on an exact set of parameter values due to nonlinearity. To mitigate this problem, global sensitivity analysis techniques are used to calculate parameter sensitivities in a wider parameter space. We applied global sensitivity analysis to a selection of five signalling and metabolic models, several of which incorporate experimentally well-determined parameters. Assuming these models represent physiological reality, we explored how the results could change under increasing amounts of parameter uncertainty. Our results show that parameter sensitivities calculated with the physiological parameter values are not necessarily the most frequently observed under random sampling, even in a small interval around the physiological values. Often multimodal distributions were observed. Unsurprisingly, the range of possible sensitivity coefficient values increased with the level of parameter uncertainty, though the amount of parameter uncertainty at which the pattern of control was able to change differed among the models analysed. We suggest that this level of uncertainty can be used as a global measure of model robustness. Finally a comparison of different global sensitivity analysis techniques shows that, if high-throughput computing resources are available, then random sampling may actually be the most suitable technique.

Introduction

Mathematical models of biochemical pathways, such as metabolism or cell signaling, are essential tools for understanding complex biological dynamics. However, these models rely on parameters—rate constants, binding affinities, and concentrations—that are frequently unknown or estimated from noisy in vitro data. Traditionally, researchers have relied on Local Sensitivity Analysis (LSA) to determine which parameters "control" the system. LSA calculates a derivative at a specific point in parameter space, providing a snapshot of system behavior.

Because biological systems are inherently nonlinear, LSA results can be fragile. If the initial parameter estimate is even slightly inaccurate, the predicted control regime might change entirely. This study addresses the reliability of model predictions under parameter uncertainty and proposes Global Sensitivity Analysis (GSA) as a more robust alternative for measuring system behavior and model stability.

Methodology

The researchers compared two primary GSA approaches against traditional local analysis across five well-established biochemical models (MAPK signaling, NFκB signaling, Yeast Cell Cycle, and Glycolysis in both S. cerevisiae and T. brucei):

  1. Derivative-based Random Sampling (Monte Carlo): Thousands to millions of parameter sets were sampled within defined ranges (e.g., ±5% to ±50% of original values). This generated probability distributions of sensitivity, revealing the likelihood of different control regimes.
  2. Optimization-based Bounds: Using Particle Swarm Optimization (PSO), the team searched for the absolute maximum and minimum sensitivity values a parameter could adopt within a given range.
  3. High-Throughput Computing: To manage the computational load, the study utilized the Condor high-throughput computing pool, leveraging approximately 3,500 CPUs to perform the massive number of required simulations.

Key Insights and Results

The Shape of Uncertainty

A central finding of the study is that the "physiological" state—the set of parameters typically used in literature—is often a statistical outlier. Sensitivity values calculated at these points were frequently not the most probable values found during random sampling. This suggests that evolution may have tuned these systems into specific, statistically unlikely corners of parameter space.

Distributions of sensitivity values for different parameters (A, B, C) showing the range of possible control regimes and the frequency of specific sensitivity outcomes under parameter uncertainty.

Multimodality and Control Flips

Many parameters exhibited multimodal sensitivity distributions. This indicates that small shifts in the environment or internal state can cause a "control flip," where a different parameter suddenly dominates the system's output. Such phenomena are invisible to LSA but are critical for understanding biological transitions and failures.

Model-Specific Robustness

  • MAPK Model: Highly sensitive; even ±5% uncertainty allowed some parameters to gain massive control, jumping from near-zero to over 180.
  • Oscillatory Models (NFκB and Cell Cycle): These systems remained relatively robust at ±10% variation, but higher uncertainty (±20%) often led to the total loss of oscillatory behavior.
  • Metabolic Models: Glycolysis models proved remarkably robust. In the T. brucei model, the glucose transporter maintained dominant control over flux in nearly all sampled conditions, though extreme uncertainty (±40%) eventually revealed alternative control peaks.

Discussion and Limitations

The authors propose that the level of parameter uncertainty required to fundamentally alter a system's "pattern of control" (e.g., shifting from a unimodal to a multimodal distribution) serves as a quantitative global measure of model robustness.

However, GSA faces the "curse of dimensionality." In complex models like the 142-parameter yeast cell cycle, even 100,000 samples provide sparse coverage of the parameter space, potentially underestimating fragility. Furthermore, this study sampled parameters independently; future work should incorporate thermodynamic and biological correlations between parameters to better reflect physiological constraints.

Conclusion

Global Sensitivity Analysis reveals that biological control is often more complex and less stable than local analyses suggest. While optimization is faster for finding worst-case scenarios, random sampling provides a richer biological picture of a system's potential states. As computational power becomes more accessible, GSA should become a standard requirement for validating the predictive power of biochemical models.

Summary updated

Abstract

Most biological models of intermediate size, and probably all large models, need to cope with the fact that many of their parameter values are unknown. In addition, it may not be possible to identify these values unambiguously on the basis of experimental data. This poses the question how reliable predictions made using such models are. Sensitivity analysis is commonly used to measure the impact of each model parameter on its variables. However, the results of such analyses can be dependent on an exact set of parameter values due to nonlinearity. To mitigate this problem, global sensitivity analysis techniques are used to calculate parameter sensitivities in a wider parameter space. We applied global sensitivity analysis to a selection of five signalling and metabolic models, several of which incorporate experimentally well-determined parameters. Assuming these models represent physiological reality, we explored how the results could change under increasing amounts of parameter uncertainty. Our results show that parameter sensitivities calculated with the physiological parameter values are not necessarily the most frequently observed under random sampling, even in a small interval around the physiological values. Often multimodal distributions were observed. Unsurprisingly, the range of possible sensitivity coefficient values increased with the level of parameter uncertainty, though the amount of parameter uncertainty at which the pattern of control was able to change differed among the models analysed. We suggest that this level of uncertainty can be used as a global measure of model robustness. Finally a comparison of different global sensitivity analysis techniques shows that, if high-throughput computing resources are available, then random sampling may actually be the most suitable technique.
BibTeX
@inproceedings{kent2013what,
  author = {Edward Kent and Stefan Neumann and Ursula Kummer and Pedro Mendes},
  title = {{What Can We Learn from Global Sensitivity Analysis of Biochemical Systems?}},
  booktitle = {PLoS},
  year = {2013},
  doi = {10.1371/journal.pone.0079244},
}
Figure