TL;DR
The authors introduce poli and poli-baselines, a unified benchmarking framework designed to clean up the "wild west" of high-dimensional Bayesian Optimization (HDBO) for discrete sequences. Their survey and experiments reveal a significant gap between current HDBO research and real-world biological needs, showing that while latent-space methods excel, simple greedy baselines remain surprisingly hard to beat in high-dimensional discrete settings.
What Problem Does This Paper Address?
Optimizing discrete sequences—like amino acids in a protein or tokens in a drug molecule—is a high-stakes challenge in the life sciences. Because evaluating these sequences (e.g., via lab experiments or complex simulations) is expensive and lacks gradient information, Bayesian Optimization (BO) is the go-to tool.
However, the field faces three major hurdles:
- The Curse of Dimensionality: Standard Gaussian Processes (GPs) struggle to fit functions as sequence lengths and category counts grow.
- Experimental Fragmentation: Researchers use wildly different initializations and evaluation budgets, making it impossible to compare "Method A" to "Method B" fairly.
- Dependency Hell: Different HDBO solvers often rely on conflicting versions of core libraries (like BoTorch or GPyTorch), making replicability a nightmare.
As shown in the figure below, most existing methods are tested on sequence lengths far shorter than those encountered in actual protein engineering or chemistry.
Methods
The authors provide two major contributions to solve these issues:
1. A Refined Taxonomy
They categorize the vast landscape of HDBO into seven distinct families: Variable selection, Additive models, Trust regions, Linear embeddings, Non-linear embeddings, Gradient information, and Structured spaces. This helps practitioners choose the right "structural assumption" for their specific problem.
2. The poli Framework
To solve the technical barriers, they developed poli (Protein Objectives Library) and poli-baselines.
- Isolation: The framework uses local network sockets and separate Conda environments to run solvers and black-box objectives. This allows a solver requiring an old library to talk to an objective requiring a new one without conflict.
- Standardization: It provides a unified NumPy-based interface for 35+ black-box tasks, ensuring consistent initialization and logging across all experiments.
Datasets
The benchmark includes:
- Ehrlich Functions: Procedurally generated oracles that mimic protein design by requiring specific "motifs" to be satisfied within a sequence.
- PMO (Practical Molecular Optimization): A suite of 23 tasks for small molecule design, covering metrics like druglikeness (QED), solubility (LogP), and docking scores.
- PestControlEquiv: A standard categorical optimization task used in the HDBO literature.
Key Insights and Ideas
- The Reality Check: Experimental setups in the field are highly heterogeneous. Figure 3/4 illustrates how initialization points and evaluation budgets vary by orders of magnitude across different papers, making published "state-of-the-art" claims difficult to verify.
- Latent Space is King: For complex chemical tasks, methods that optimize in a learned, continuous latent space (via VAEs) significantly outperform those attempting to navigate the discrete sequence space directly.
- Simplicity Wins (Sometimes): In low-budget scenarios, simple greedy baselines like Directed Evolution often perform as well as, or better than, sophisticated BO methods. This echoes recent skepticism in the field regarding the added value of complex surrogate models for certain molecular tasks.
Key Results
The authors benchmarked several top-tier solvers, including TuRBO, SAASBO, BAxUS, and Bounce.
- Sequence Design: On Ehrlich functions, performance dropped sharply as sequence length ($L$) increased. While
Bounceperformed well on simpler tasks, it struggled to scale to $L=64$. - Molecular Optimization: In the PMO benchmark (128D latent space), continuous optimizers like
SAASBOandCMA-ESshowed a clear advantage. However, many advanced solvers likeProbRepandBounceran into significant memory or runtime issues on larger problems.
Limitations and Future Work
The authors acknowledge that their current black-box oracles are mostly data-driven or closed-form, which may not fully capture the noise and complexity of real-world physical simulations. They also note that they have only benchmarked a subset of the methods identified in their taxonomy.
Future work will focus on expanding poli to include more high-fidelity biological objectives and conducting a more systematic analysis of the "myriad of design choices" (like kernel selection and hyperparameters) that influence HDBO performance.
Summary updated