Match the Geometry Before the Surrogate: Extreme Low-Budget Optimization in High Dimensions

Michal Průšek, Adam Novozámský, Filip ŠroubekarXiv 2026 · arXiv

In the world of drug discovery and protein design, a "budget" isn't just a number on a spreadsheet—it’s the number of physical experiments a wet-lab can perform in a single round. While many machine learning benchmarks for high-dimensional Bayesian Optimization (BO) assume hundreds or thousands of evaluations, a realistic lab cycle might only allow for 30.

A new paper from researchers at the Czech Technical University and the Czech Academy of Sciences, titled "Match the Geometry Before the Surrogate," argues that in this extreme low-budget regime, we’ve been focusing on the wrong thing. It’s not the complexity of your surrogate model (like a Gaussian Process) that matters most; it’s the geometry of where you choose to look.

TL;DR

In extreme low-budget optimization ($N=30$), the shape of the search space matters more than the surrogate model. By restricting candidate generation to the "spherical shell" where VAE encoder means actually concentrate, simple search strategies can match or outperform sophisticated Bayesian Optimization methods.

What Problem Does This Paper Address?

Molecular optimization often happens in the "latent space" of a Variational Autoencoder (VAE). We map discrete molecules to continuous vectors, use BO to find "better" vectors, and decode them back into new molecules.

The authors identify a critical mismatch: most BO methods propose new candidates within axis-aligned boxes or trust regions. However, in high-dimensional VAEs with Gaussian priors, the encoded "means" of real molecules don't fill a box—they concentrate on a narrow spherical shell. If your optimizer is looking in the corners of a high-dimensional box, it’s searching in "empty space" where the decoder has never seen data, leading to poor molecular generation.

Methods

The authors introduce the on-sphere scaffold, a constraint that forces the optimizer to only propose candidates on the empirical shell occupied by known molecules. To test the power of this geometric constraint, they developed two "minimalist" optimizers:

  1. GEOWALK: A surrogate-free baseline. It performs uniform exploration on the shell, followed by a "geodesic local search" (a random walk that stays on the sphere) starting from the best point found.
  2. GEOCMA: Adapts the well-known CMA-ES algorithm to work within a random low-dimensional subspace, while still projecting candidates onto the spherical shell.

They also conducted a factorial experiment using a standard Gaussian Process (GP) to isolate two variables: the support (searching in a box vs. on the shell) and the acquisition maximizer (picking the best from a random set vs. using a gradient-based continuous optimizer like L-BFGS-B).

Datasets

  • GuacaMol: A standard benchmark for goal-directed molecular generation (11 tasks used).
  • ZINC-250k: A reference set of 250,000 molecules used to characterize the VAE's latent geometry.
  • LassoBench: A high-dimensional sparse regression benchmark.
  • NAS-Bench-101: A neural architecture search benchmark.

Key Insights and Ideas

  • The "Shell" is Real: In a 256-dimensional VAE, the typical norm of a sample from the prior is $\approx 16$, but the actual encoded molecules have a mean norm of $\approx 8$. They live in a thin, mid-radius shell.
  • Surrogates are Superfluous (at $N=30$): Surprisingly, after 30 evaluations, the surrogate-free GEOWALK performed just as well as a GP-based optimizer on the shell. The GP is helpful early on (around 10-20 evals), but the geometric constraint does most of the heavy lifting by the end of the budget.
  • The Interaction Effect: The benefit of searching on the shell is most pronounced when using "candidate-set maximization" (scoring a bunch of random points and picking the best). If you use a continuous optimizer like L-BFGS-B, it can sometimes "climb" out of the box and find the shell anyway, making the explicit constraint less vital.

Key Results

The authors found that GEOWALK, despite its simplicity, outperformed several established high-dimensional BO methods that search in boxes, such as TuRBO and BAxUS.

Table 1: Cross-task means and paired statistics for GEOWALK versus all comparators at N = 30 on one matched set of n = 440 (task, seed) cells of the 11 GuacaMol tasks. REMBO and HeSBO propose on the empirical shell; GaBO and Riemannian Matérn GP propose on the sphere manifold; all other comparators propose off the shell. ∆is the improvement of GEOWALK over the comparator. W/L counts the matched cells where GEOWALK wins or loses, excluding ties. ptask is the exact two-sided Wilcoxon test on the 11 per-task mean differences, not corrected. d is Cohen’s d of the paired cell differences. Bold marks the highest mean. Notes are in Appendix H.

In their factorial experiment (Figure 3), they demonstrated that the "Empirical shell" support consistently yielded higher scores than "Box" support when using candidate-set argmax.

Table 3: The support × acquisition-maximization factorial: cross-mean of each cell. One Matérn-5/2 GP (DSP prior) on n = 440 matched cells; continuous maximization is L-BFGS-B on both supports. Cross-means as in Table 1. The five contrasts of these cells, with their task counts and corrected p values, are in Table 19 (Appendix J).

Limitations and Future Work

  • Scope of VAEs: The results are specific to Gaussian-prior VAEs. The authors note that other architectures (like Chembed) might not exhibit the same "shell" advantage.
  • Budget Scaling: The "surrogate-free" advantage likely disappears as the budget increases toward 100 or 1,000 evaluations, where the GP has enough data to build a truly superior model.
  • Practical Lesson: The authors conclude with a warning: before claiming a new surrogate model is "better," researchers must control for the candidate support and the acquisition maximizer. A gain attributed to a fancy new GP might actually just be the result of accidentally looking in a better part of the geometry.

For engineers and researchers, the message is clear: if you're working with a tiny experimental budget, stop tweaking your kernel hyperparameters and start looking at the "norm" of your data. You might find that your optimizer is simply looking in the wrong neighborhood.

Summary updated

Abstract

Low-budget molecular optimization is often framed as Bayesian optimization in a learned latent space, but standard benchmarks typically allow hundreds or thousands of evaluations, whereas a realistic wet-lab round may allow only a few dozen. We revisit this regime using a SELFIES molecular variational autoencoder (VAE) with a Gaussian prior and GuacaMol objectives at N = 30 evaluations. The main finding is not a new surrogate model, but an interaction in candidate generation. The encoder means of a reference set of molecules concentrate near a narrow spherical shell, while many high-dimensional optimizers propose candidates from boxes or box-shaped trust regions. We study a simple constraint, the on-sphere scaffold, that restricts candidate generation to this shell. We then introduce two minimal optimizers to probe what remains useful once the support is fixed. GeoWalk performs geodesic local search on the shell without a surrogate. After 30 evaluations, it shows no detected difference from a Gaussian-process (GP) surrogate on the shell, which is ahead earlier in the budget. GeoCMA adds covariance learning in a random subspace. It does not improve on GeoWalk in the molecular benchmark. On LassoBench it has a lower mean loss than GeoWalk, but full-dimensional CMA-ES remains stronger on most tasks. A factorial experiment with a single GP crosses the candidate support (shell or box) with the acquisition maximizer (a finite candidate set or a continuous optimizer). In this benchmark, shell candidates outperformed box candidates when acquisition was maximized over a candidate set, but not with continuous maximization by L-BFGS-B. These results come from a controlled study of one Gaussian-prior VAE benchmark and do not recommend VAEs for molecular optimization. The practical lesson is to control the candidate support and the acquisition maximizer before attributing a gain to the surrogate.
BibTeX
@misc{prusek2026match,
  author = {Michal Průšek and Adam Novozámský and Filip Šroubek},
  title = {{Match the Geometry Before the Surrogate: Extreme Low-Budget Optimization in High Dimensions}},
  year = {2026},
  eprint = {2611.10267},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.10267},
}
Figure