A bench biologist who wants a better enzyme or a stronger DNA enhancer faces two walls at once. The first is mathematical: the assay that tells her whether a sequence works — a fluorescence plate reader, a stability measurement, a proprietary cloud model — gives only a number per candidate, with no gradients, and each "query" costs real money and days of waiting. The second is computational: the methods that could make those few queries count, borrowed from black-box adversarial optimization, live in research codebases that assume fluency in PyTorch, GPU clusters, and hyperparameter tuning. Meanwhile, the tools that biologists can actually use — point-and-click transcriptomics platforms like PIVOT, cloud analysis servers like WebMeV, and the LLM-driven analysis agents of CellAgent — are strictly evaluative. They cluster, annotate, and visualize data that already exists; none of them designs new sequences.
This proposal claims the intersection is worth occupying for a scientific reason, not just a usability one. In adversarial black-box optimization, three mechanisms are known to cut query budgets by multiples: biasing search directions toward a surrogate model's gradient (Guessing Smart, PRGF), restricting search to a subspace spanned by several reference models' gradients (Subspace Attack), and decaying the surrogate prior adaptively as it loses alignment with the true landscape (PRGF). None of these has been transplanted into biological sequence design, where query budgets are smallest and the payoff largest. The closest biological systems each stop short: LatentDE uses a single surrogate's gradient as a one-shot warm-up and then abandons gradient guidance entirely; P3BO runs a portfolio of optimizers but treats each as an opaque proposal generator, never pooling their gradients; GLID²E optimizes against differentiable oracles it fully controls rather than an uninspected assay; and TS-DE proves query-efficiency guarantees but only under a linear map from sequence motifs to fitness, which rules out the epistasis that Enformer shows dominates real regulatory sequence function. We propose to build the missing system: an adaptive, multi-surrogate gradient-prior optimizer for discrete biological sequences, delivered behind a conversational interface that a non-programmer can drive with a spreadsheet of their own measurements.
Core Claim
The central scientific claim is that a search direction for black-box biological sequence design should be a dynamically reweighted mixture of what an ensemble of surrogate models believes and what the assay has actually revealed, and that this mixture weight can be estimated online from the agreement between predicted and observed fitness changes. Concretely, at optimization round $t$ we estimate the uphill direction as
$$\hat{g}_t = (1-\lambda_t)\,\bar{g}^{\text{ens}}_t + \lambda_t\,\hat{g}^{\text{ZO}}_t,$$
where $\bar{g}^{\text{ens}}_t$ is the average gradient, in the latent space of a pre-trained sequence model, of an ensemble of surrogates fine-tuned on the user's measurements so far; $\hat{g}^{\text{ZO}}_t$ is a zeroth-order gradient estimate — a finite-difference over the last batch of assayed candidates, in the spirit of ZOO; and $\lambda_t \in [0,1]$ grows as the estimated alignment between surrogate predictions and observed assay returns falls, mirroring how PRGF monitors cosine similarity to decide when a transfer prior has perished. This directly addresses the documented fragility of LatentDE, whose authors identify their single, possibly miscalibrated fitness predictor as the method's bottleneck, and it does so with the ensemble-subspace idea that Subspace Attack showed effective even when reference models were trained on only 2,000 images — a regime very close to a wet-lab campaign's first batch of measurements.
Two secondary mechanisms complete the transplant. First, mutations are proposed within a low-dimensional subspace spanned by the ensemble's gradients rather than uniformly across the sequence, the discrete analogue of subspace restriction, which matters because a 237-residue protein has $20^{237}$ neighbors and uniform sampling is hopeless. Second, we exploit structure priors specific to biology: motifs and conserved blocks are the sequence analogue of the spatial tiling prior that let BanditsTD cut average queries from 1,735 to 703 and failure rates from 22.2% to 4.6% on ImageNet, and the near-collision of successive gradients (cosine similarity $\approx 0.92$ in their measurements) becomes a momentum prior across design rounds.
Research Program
The work has three components of unequal weight. The optimization engine is the scientific core; the accessibility layer is the delivery vehicle and a genuine design constraint (it forces the method to work with messy, small, user-supplied data rather than clean benchmark oracles); the analysis component connects our mechanism to existing theory.
For the engine, we will build a campaign loop in which the user supplies a starting sequence, a natural-language goal, and each round's assay results; the system fine-tunes a small ensemble of pre-trained models (ESM-2 for proteins, an Enformer-style model for DNA) on the accumulated measurements, computes latent-space gradients for each ensemble member, and returns a ranked batch of proposed mutants with per-position explanations. The key engineering problem is estimating $\lambda_t$ reliably from very few observations — tens, not thousands — so we will develop and compare estimators based on rank correlation between predicted and realized fitness deltas across each batch, and study how quickly they detect the surrogate misalignment that PRGF showed decays as the search leaves its starting distribution.
For the accessibility layer, we follow the agentic pattern of CellAgent — an LLM that plans, writes, and executes analysis steps from conversational intent — but point it at the design loop instead of the analysis loop: the agent ingests the user's goal and results spreadsheet, invokes the optimization engine, and translates its proposals back into biological language. This is a deliberate extension, not a reimplementation: CellAgent's own scope is annotation and visualization, and nothing in the accessible-tooling literature — from miRPlant's consolidated desktop executable to AMULETY's unified embedding API — closes a generative design loop.
For analysis, we will examine whether the ensemble-latent gradient can be read as a learned nonlinear feature map that repairs the linearity assumption under which TS-DE obtains its $\tilde{O}(d^2 \sqrt{MT})$ regret bound for Thompson-sampling-guided directed evolution — a theoretical bridge rather than a target, since TS-DE itself concedes its linear model omits epistatic interactions that Enformer demonstrates are essential.
Baselines
The naive reference point is conventional directed evolution: random mutagenesis of the starting sequence followed by screening of each batch. It requires no modeling, and it is what a bench scientist does today; any method that cannot beat it per query has no practical case. TS-DE documents that plain directed evolution stalls on fitness plateaus and is highly sensitive to mutation rate, and their bandit-guided variant achieved a 30+ fold improvement in CRISPR guide efficiency in real lab tests, which suggests the bar is real but movable.
The strongest existing competitor on the sequence-design benchmarks is P3BO, a population-based portfolio that adaptively allocates experimental capacity among evolutionary strategies, model-based optimizers, and generative models. It earned the highest average rank across the 105-landscape benchmark suite covering TfBind transcription-factor binding, UTR expression, and PfamHMM scoring — no reported result found on Lacuna for its exact numerical scores on those landscapes — and its authors specifically demonstrated that single optimizers are brittle across landscapes, with rankings inverting between tasks. We will run P3BO with the same starting sequences, the same per-round batch sizes (mimicking 96-well plate constraints), the same training data for its model-based components, and its own adaptive hyperparameter tuning enabled so that tuning budgets are matched. Beating P3BO means achieving higher fitness at fixed cumulative query counts — measured as the area under the fitness-versus-query curve and as queries needed to reach a target fitness threshold — across the 105-landscape suite, and holding that advantage when the batch size is capped at realistic wet-lab scales.
The strongest competitor on the DNA-enhancer and protein-stability settings is GLID²E, a PPO-tuned discrete diffusion approach that reported a predicted enhancer activity of 7.29 versus 5.61 for DRAKES on the ~700,000-sequence Enformer-oracle benchmark, a 98.4% chromatin accessibility score, and a 76.7% success rate on the ~19,700-structure Megascale stability dataset. Those numbers were obtained in a white-box setting where the oracle is fully differentiable and queries are effectively free; to make the comparison fair we will rerun GLID²E's public configuration under our black-box protocol, giving it the same query budget, the same per-round batch limits, and the same surrogate-training data, and we count ourselves as beating it only if we surpass its fitness at matched query counts — with its reported 7.29 and 76.7% serving as the reference points its configuration achieves when unconstrained. We will additionally run pure zeroth-order NES-style search, since BanditsTD quantified exactly how much such methods leave on the table in vision (1,735 average queries versus 703 with priors), and a LatentDE-style static warm-up followed by noise-sampling evolution as the direct predecessor of our mechanism.
Finally, two ablations isolate the proposal's key ingredients: fixing $\lambda_t$ to a constant (removing adaptive prior decay, leaving the ensemble subspace intact) and collapsing the ensemble to a single surrogate (removing the subspace, leaving adaptation intact). If the adaptive-blend ablation loses most of the gain, the core claim is wrong and the interesting part is only the ensemble; the experiment is designed so that either outcome is informative.
Why not just…?
Why not just train one strong offline model and do gradient ascent locally? This is the zero-query transfer baseline, and it is the most dangerous alternative because when the user's assay happens to match a pre-trained oracle closely it will win outright — we will run this regime explicitly and report it. The evidence against it as a general strategy is substantial: P3BO documented that individual offline models suffer catastrophic brittleness across landscapes, with rankings inverting between tasks; PRGF showed theoretically and empirically that surrogate gradients decay in usefulness as the search moves away from its starting point; and GLID²E observed that unconstrained optimization against an offline model rewards biologically non-viable "gibberish" sequences. Our adaptive $\lambda_t$ is precisely the mechanism that detects when this alternative is failing and falls back to assay feedback; the honest framing is that pure transfer is a special case of our method at $\lambda_t = 0$, and the experiments will show how often that special case is safe.
Why not just use standard zeroth-order optimization (NES or random search)? It requires no surrogate machinery at all. But the query arithmetic is brutal in a wet lab, where a round of screening means days and dollars. In vision, unbiased zeroth-order search needed an average of 1,735 queries where prior-guided search needed 703, and in decision-based settings unbiased boundary search reached only a 33% success rate within 15,000 queries against 85% with surrogate-biased sampling, per BanditsTD and Guessing Smart respectively. We will include NES as a baseline under identical budgets; if it matches our method on the 105-landscape suite at 96-candidate batches, the proposal's practical case collapses, and we will say so in the results.
Why not just wrap an existing optimizer like P3BO or LatentDE in an LLM chat interface? This is the closest competing route to the accessibility half of the proposal and deserves a direct answer. It would deliver conversational access without any new optimization science, and it would inherit exactly the fragilities the engine is designed to fix: a single possibly-miscalibrated surrogate in the LatentDE case, and portfolio members that never share gradient information in the P3BO case. The scientific contribution of this proposal is the adaptive multi-surrogate prior in discrete sequence space; the conversational layer is how that mechanism reaches the users who need it and, importantly, how it gets fed the small, messy, user-owned datasets it is designed around. A chat wrapper around a static optimizer cannot reduce query counts, and query counts are the binding constraint in every wet-lab campaign in the file.
Why not just backpropagate through a differentiable oracle end-to-end? Where the target is a model the user fully controls, this is legitimate and strong. GLID²E already showed that decoupled policy optimization trains roughly 1.7× faster than backprop-through-diffusion (13.54s versus 23.28s per epoch) with lower memory, and physical assays, proprietary APIs, and multi-scale simulators are simply not differentiable. Our protocol keeps the differentiable-oracle setting as a diagnostic ceiling and evaluates everything under black-box budgets.
Proposed Experiments
The main evaluation runs on the P3BO benchmark suite of 105 landscapes (TfBind, UTR, PfamHMM), treating each landscape's ground-truth scorer as a black-box assay with per-round batch caps of 96 candidates, reporting area under the fitness-versus-query curve and queries to target fitness against P3BO, NES, the static-warm-up variant, and pure transfer. A second track evaluates on the LatentDE protein benchmarks — avGFP (237 amino acids) plus AAV, TEM-1 β-lactamase, E4B, AMIE, LGK, Pab1, and UBE2I — where LatentDE reported statistically significant outperformance on avGFP and superiority across all eight datasets in maximum fitness, though no reported result found on Lacuna for its exact numerical scores; we therefore rerun LatentDE ourselves under matched budgets rather than comparing to unreported numbers. A third track uses the GLID²E settings: the ~700,000-sequence Enformer-oracle enhancer dataset and the ~19,700-structure Megascale stability set, with GLID²E's reported 7.29 predicted activity and 76.7% stability success rate as the unconstrained reference points and our black-box reruns as the fair comparison. Because SoK: Pitfalls in Evaluating Black-Box Attacks warns that black-box benchmarks inflate results through hyperparameter leakage and unequal query limits, we will fix all query budgets, batch sizes, and tuning budgets before running, publish the protocol, and report failures as well as successes. A hard-label variant — assays that return only pass/fail — will be tested with the biased decision-boundary machinery of Guessing Smart, which reached an 85% success rate within 15,000 queries in vision, adapted to sequence space.
The decisive checks are three. First, the adaptation curve: plot estimated surrogate–assay alignment and $\lambda_t$ over rounds, and show that fitness gains track the adaptive schedule rather than any fixed blend; this is the direct test of the perishability hypothesis from PRGF in a biological domain. Second, the data-scaling curve: performance as a function of measurements supplied, testing whether the Subspace Attack finding that reference models trained on just 2,000 images still help carries over to ensembles fine-tuned on a few hundred assay results. Third, a usability study in the spirit of the accessible-tools literature: non-programmer biologists given a design goal, a conversational interface to our system, and a command-line path to the same engine, measured on task completion, time to first valid proposal batch, and correctness of the campaign they configure. If the engine wins on queries but the interface does not win on usability, the accessibility claim fails on its own terms — and the study is built to reveal that rather than hide it.
References
- PIVOT: platform for interactive analysis and visualization of transcriptomics data
- WebMeV: A Cloud Platform for Analyzing and Visualizing Cancer Genomic Data
- CellAgent: LLM-Driven Multi-Agent Framework for Natural Language-Based Single-Cell Analysis
- Guessing Smart: Biased Sampling for Efficient Black-Box Adversarial Attacks
- Query-Efficient Black-Box Adversarial Attacks Guided by a Transfer-Based Prior
- Subspace Attack: Exploiting Promising Subspaces for Query-Efficient Black-box Attacks.
- LatentDE: Latent-based Directed Evolution accelerated by Gradient Ascent for Protein Sequence Design
- Population-Based Black-Box Optimization for Biological Sequence Design
- Bandit Theory and Thompson Sampling-Guided Directed Evolution for Sequence Optimization
- Effective gene expression prediction from sequence by integrating long-range interactions
- ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models
- Prior Convictions: Black-box Adversarial Attacks with Bandits and Priors
- miRPlant: an integrated tool for identification of plant miRNA from RNA sequencing data
- AMULETY: A Python package to embed adaptive immune receptor sequences
- SoK: Pitfalls in Evaluating Black-Box Attacks