Bayesian Optimization for Design of Multiscale Biological Circuits

Charlotte Merzbacher, Oisin Mac Aodha, Diego A. Oyarzún

TL;DR

Designing complex biological circuits is computationally challenging due to their multiscale nature and the need to optimize both architecture and parameters. This paper introduces a Bayesian optimization approach that efficiently navigates these complex design spaces, enabling faster and more robust design of synthetic biological systems.

What Problem Does This Paper Address?

Synthetic biology aims to engineer biological circuits for various functions, from gene regulation to metabolic pathways. However, designing these circuits is difficult because they often involve components operating at vastly different timescales and concentrations (multiscale systems). Current computational optimization methods struggle with these systems because their models are "numerically stiff," leading to extremely slow simulations. This slowness makes it infeasible to explore the vast design space, which includes not only continuous parameters (like binding rates) but also discrete choices about the circuit's architecture (its "wiring diagram"). The paper addresses the need for an efficient and scalable method to optimize both the architecture and parameters of these complex, multiscale biological circuits.

Methods

The core of this paper is the application of Bayesian Optimization (BayesOpt) to the problem of designing synthetic biological circuits. BayesOpt is a machine learning technique that treats the objective function (e.g., circuit performance) as a random variable. It iteratively builds a statistical model of this function's "landscape" based on previous evaluations. This model is then used to intelligently select the next point in the design space to evaluate, balancing exploration of unknown regions with exploitation of promising areas.

What's novel here is the adaptation of BayesOpt to a mixed-integer optimization problem. This means the algorithm simultaneously optimizes:

  • Discrete variables: The circuit's architecture (e.g., different feedback loop configurations).
  • Continuous variables: The kinetic parameters of the circuit components (e.g., binding affinities, enzyme expression rates).

The authors implemented this using the Python HyperOpt package, employing Expected Improvement as the acquisition function and a Tree-structured Parzen Estimator (TPE) as the nonparametric model. They integrated constraints directly into the search space and used ODE models to simulate circuit behavior and compute the objective function.

Datasets

The paper does not introduce new datasets for training. Instead, it uses existing models of biological pathways from the literature for evaluation. These include:

  • A "toy" metabolic pathway model.
  • A glucaric acid production pathway in E. coli.
  • A fatty acid synthesis pathway in E. coli.
  • A p-aminostyrene production pathway in E. coli.

These models, along with their parameters and equations, are described in the Supporting Information.

Key Insights and Ideas

  • BayesOpt excels at multiscale systems: The numerical stiffness of ODE models for multiscale biological systems makes traditional optimization methods computationally prohibitive. BayesOpt's ability to intelligently select evaluation points significantly overcomes this bottleneck.
  • Joint optimization of architecture and parameters is crucial: The paper demonstrates that optimizing only parameters within a fixed architecture, or vice versa, is insufficient. The interplay between architectural choices and parameter values creates a highly non-convex landscape that requires a method capable of navigating both discrete and continuous spaces simultaneously.
  • Robustness to uncertainty is achievable: The method can be used to assess how well an optimal design performs under parameter uncertainty (e.g., variations in enzyme kinetics), a critical factor for experimental implementation.
  • Balancing multiple objectives is key: For complex systems like metabolic pathways, designers often face trade-offs between production yield, cost of expression, speed, and accuracy. BayesOpt can effectively explore these trade-off curves.

Key Results

  • Speed and Efficiency: BayesOpt was significantly faster than other methods tested (random search, grid search, genetic algorithms, gradient-based methods) for the toy model, achieving optimal solutions in an average of 27 seconds per run. It provided a ~30-fold improvement over a genetic algorithm.
  • Robustness: For the glucaric acid pathway, perturbing kinetic parameters by ±100% did not significantly affect the optimal objective function value or the chosen architecture (dual control was selected in >85% of runs).
  • Scalability: The method successfully optimized a complex p-aminostyrene pathway model with 23 ODEs, 27 architectures, and 16 continuous parameters, achieving a low objective function value with a mean run time under 2 minutes.
  • Architecture Selection: In several cases, the "dual control" architecture emerged as optimal, highlighting its effectiveness in balancing competing demands. However, for the p-aminostyrene pathway, no single architecture was consistently optimal across multiple runs, indicating the presence of multiple near-optimal solutions.

Limitations and Future Work

  • Model Dependency: The accuracy of the optimization is dependent on the quality and completeness of the underlying ODE models. Constructing these models requires significant domain knowledge regarding enzyme kinetics and metabolic processes.
  • Computational Cost for Very Large Systems: While efficient, optimizing extremely large and complex systems might still require substantial computational resources.
  • Future Work: The authors suggest that this method could be extended to fit temporal circuit dynamics to experimental data, discern which discrete circuit mechanisms best match observed behavior, and be applied to a broader range of synthetic biology design tasks beyond metabolic engineering. They also note the growing interest in machine learning for metabolic engineering and see their work as contributing to this trend.

Summary updated

Abstract

Recent advances in synthetic biology have enabled the construction of molecular circuits that operate across multiple scales of cellular organization, such as gene regulation, signaling pathways, and cellular metabolism. Computational optimization can effectively aid the design process, but current methods are generally unsuited for systems with multiple temporal or concentration scales, as these are slow to simulate due to their numerical stiffness. Here, we present a machine learning method for the efficient optimization of biological circuits across scales. The method relies on Bayesian optimization, a technique commonly used to fine-tune deep neural networks, to learn the shape of a performance landscape and iteratively navigate the design space toward an optimal circuit. This strategy allows the joint optimization of both circuit architecture and parameters, and provides a feasible approach to solve a highly nonconvex optimization problem in a mixed-integer input space. We illustrate the applicability of the method on several gene circuits for controlling biosynthetic pathways with strong nonlinearities, multiple interacting scales, and using various performance objectives. The method efficiently handles large multiscale problems and enables parametric sweeps to assess circuit robustness to perturbations, serving as an efficient <i>in silico</i> screening method prior to experimental implementation.
BibTeX
@inproceedings{merzbacher2023bayesian,
  author = {Charlotte Merzbacher and Oisin Mac Aodha and Diego A. Oyarzún},
  title = {{Bayesian Optimization for Design of Multiscale Biological Circuits}},
  booktitle = {ACS},
  year = {2023},
  doi = {10.1021/acssynbio.3c00120},
}
Figure