Information-based Variational Model Reduction of high-dimensional Reaction Networks

Markos A. Katsoulakis, Pedro Vilanova

Abstract

In this work we present new scalable, information theory-based variational methods for the efficient model reduction of high-dimensional deterministic and stochastic reaction networks. The proposed methodology combines, (a) information theoretic tools for sensitivity analysis that allow us to identify the proper coarse variables of the reaction network, with (b) variational approximate inference methods for training a best-fit reduced model. This approach takes advantage of both physicochemical modeling and data-based approaches and allows to construct optimal parameterized reduced dynamics in the number of variables, reactions and parameters, while controlling the information loss due to the reduction. We demonstrate the effectiveness of our model reduction method on several complex, high-dimensional chemical reaction networks arising in biochemistry.

Introduction

In systems biology and chemical engineering, researchers frequently encounter reaction networks involving thousands of distinct species and complex molecular interactions. Modeling these systems presents a dual challenge: the computational cost of simulating every interaction is often prohibitive, and the "curse of dimensionality" makes traditional sensitivity analysis—identifying which parameters actually drive system behavior—nearly impossible as models grow.

Existing reduction techniques, such as timescale separation or lumping, often require extensive manual expertise or result in "black box" models that strip away the original biochemical meaning. This paper introduces a scalable, information-theoretic framework designed to simplify massive networks while preserving their physical interpretability. By bridging the gap between physical modeling and machine learning, the proposed method identifies critical species and reactions to create reduced models that are both accurate and orders of magnitude faster to simulate.

Methodology

The authors propose a six-step pipeline that integrates pathwise sensitivity analysis with variational inference. The framework is built on several key pillars:

Pathwise Fisher Information Matrix (pFIM)

Rather than assessing how a single output changes with respect to a parameter, the authors utilize the pFIM to measure the sensitivity of the entire system trajectory. A critical advantage of this approach for reaction networks is that the pFIM is sparse and block-diagonal. This structure allows the matrix to be computed in linear time relative to the number of parameters, providing a significant speedup over classical adjoint methods.

Parameter and Variable Selection

Parameters are ranked based on their "information contribution." By retaining only the top parameters—which typically account for 95-99% of the total information—the authors use the network's stoichiometry to identify the specific chemical species and reactions associated with those parameters. This ensures the reduced model remains grounded in the original biochemistry.

Variational Model Reduction and Force Matching

Model reduction is framed as an optimization problem. The goal is to find a reduced model that minimizes the Relative Entropy (Kullback-Leibler divergence) between the path distribution of the full model and the reduced one. To achieve this, the authors derive a simplified loss function that enables training via "Dynamic Force Matching," a technique adapted from molecular dynamics that uses time-series data from the full model to optimize the reduced parameters.

Experimental Validation

The methodology was validated against several complex biochemical models from the EMBL-EBI BioModels database, spanning both deterministic and stochastic regimes:

  • Protein Homeostasis Network: A model featuring 52 species and 80 reactions. The system operates in a "sloppy" regime where most parameters have negligible effects. The framework successfully reduced this to 12 species and 10 parameters while maintaining nearly identical mean-field trajectories.
  • Mammalian Circadian Clock: A 16-species, 52-reaction model exhibiting non-linear oscillations. The method captured the oscillatory period and amplitude accurately even after eliminating 17 parameters and 21 reactions.
  • Epidermal Growth Factor Receptor (EGFR): A 24-species signaling model used to evaluate transient dynamics.
  • Stochastic Pure Jump Models: The authors applied the method to stochastic versions of the Circadian and EGFR models. Even in non-Gaussian regimes with small particle counts, the reduced models accurately captured the probability distributions of species counts.

Key Insights

The Utility of Sloppiness

Complex biochemical models are often "sloppy," meaning their macroscopic behavior is determined by only a few "stiff" parameter combinations. The pFIM serves as an efficient tool for isolating these combinations in high-dimensional spaces, allowing for massive compression without sacrificing predictive power.

Information Geometry and Structure Preservation

By focusing on "path-space" (the space of all possible time-series), the method captures temporal correlations that static sensitivity analyses often miss. Furthermore, because the reduced model is constrained to be a smaller reaction network rather than a generic neural network, the variables remain physical concentrations, ensuring the results are useful for experimentalists.

Scalability

The computational requirements for the sensitivity analysis scale linearly with the number of parameters. This makes the framework feasible for "Big Data" models where traditional methods fail due to computational complexity.

Limitations and Future Directions

While the framework is robust, it is currently optimized for specific time intervals $[0, T]$. A model reduced for short-term transients may not be optimal for long-term steady states. Additionally, while the method performs well in stochastic systems, validation often relies on mean-field comparisons, which may overlook rare events in very small populations. Future work aims to integrate this approach with Global Sensitivity Analysis to further enhance the robustness of the reductions across broader parameter spaces.

Summary updated

Abstract

In this work we present new scalable, information theory-based variational methods for the efficient model reduction of high-dimensional deterministic and stochastic reaction networks. The proposed methodology combines, (a) information theoretic tools for sensitivity analysis that allow us to identify the proper coarse variables of the reaction network, with (b) variational approximate inference methods for training a best-fit reduced model. This approach takes advantage of both physicochemical modeling and data-based approaches and allows to construct optimal parameterized reduced dynamics in the number of variables, reactions and parameters, while controlling the information loss due to the reduction. We demonstrate the effectiveness of our model reduction method on several complex, high-dimensional chemical reaction networks arising in biochemistry.
BibTeX
@misc{katsoulakis2018information,
  author = {Markos A. Katsoulakis and Pedro Vilanova},
  title = {{Information-based Variational Model Reduction of high-dimensional Reaction Networks}},
  year = {2018},
  eprint = {1807.05319},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1807.05319},
}

Loading related content...

Figure