Introduction
Bayesian Optimization (BO) is a powerful framework for optimizing expensive black-box functions. In computational biology, it is frequently used to estimate parameters for complex metabolic models. However, standard BO faces a significant computational bottleneck: the Gaussian Process (GP) surrogate models typically used have a computational cost that scales cubically ($O(n^3)$) with the number of observations. As biological simulations become faster and datasets grow larger, the optimizer itself can become slower than the model it aims to tune. Ulmasov et al. propose the Dimension Scheduling Algorithm (DSA) to overcome these limitations in high-dimensional biological systems.
The Dimension Scheduling Algorithm (DSA)
The core challenge addressed by DSA is maintaining the efficiency of BO as the number of parameters (dimensions) and data points increases. Instead of modeling the entire high-dimensional space with a single, computationally expensive GP, DSA employs several strategies to streamline the process:
- Dimension Subsets and Scheduling: At each iteration, DSA selects a small subset of dimensions to optimize while keeping others fixed at their best-known values.
- PCA-based Importance Sampling: To ensure the most impactful parameters are prioritized, the algorithm uses Principal Component Analysis (PCA) on the accumulated data. Dimensions corresponding to higher eigenvalues are scheduled for optimization more frequently.
- Distributed Gaussian Processes: Rather than one global GP, DSA utilizes multiple smaller GPs, each trained on data specific to a subset of dimensions. This significantly reduces the training overhead and keeps the $O(n^3)$ cost manageable.
Unlike previous methods like Additive BO, DSA does not assume the objective function is strictly decomposable, making it better suited for the highly interconnected parameter spaces found in biological systems.
Experimental Evaluation
The researchers evaluated DSA using ten different model variants of microalgae metabolism based on the DRUM framework. These models involve 10 to 12 dimensions, where the goal is to minimize the squared error between simulations and experimental data for biomass, lipids, and chlorophyll. DSA was benchmarked against standard BO over 500 iterations.
Results and Discussion
The experimental results demonstrate that DSA provides a substantial improvement in both computational efficiency and optimization performance.
Computational Efficiency
DSA achieved a dramatic reduction in total execution time compared to standard BO. By utilizing smaller, distributed GPs and optimizing subsets of dimensions, the mathematical overhead of the optimizer was slashed.
As shown in the figure above, DSA consistently completed optimization tasks in approximately one-fifth of the time required by traditional BO across all tested models (M19 through M35).
Optimization Performance
Beyond speed, DSA also proved more effective at finding lower objective values. Standard BO often struggles in high-dimensional spaces as the GP becomes cumbersome and exploration slows down. In contrast, DSA's "slicing" approach allows it to explore the parameter landscape more rapidly.
The convergence plots indicate that DSA (solid lines) typically reaches lower objective values faster than standard BO (dashed/dotted lines). In nearly all tested microalgae models, DSA outperformed the benchmark within the same iteration limit, demonstrating its robustness for complex biological parameter estimation.
Conclusion
The Dimension Scheduling Algorithm (DSA) effectively addresses the scalability issues of Bayesian Optimization in high-dimensional biological contexts. By focusing on impactful dimension subsets and reducing GP complexity, DSA offers a 5x speedup and superior accuracy over standard methods. Future work may explore hybrid approaches that combine DSA's rapid exploration with gradient-based local refinement to further polish final results.
Summary updated