Knob-Free Multi-Objective Protein Design: Entropy Weighting under a Nonmonotone Line Search

A biologist using a protein-design platform today has to set two numbers they are poorly equipped to set: the trade-off weights that combine conflicting property scores (activity against stability, say, or immunogenicity) and the learning rate that controls how fast the optimizer moves. Accessible platforms resolve this in one of two unsatisfying ways. They freeze the defaults, which InstantDL: an easy-to-use deep learning pipeline for image segmentation and classification explicitly documents as sacrificing peak performance, or they hand the problem to an automated hyperparameter search, which AutoProteinEngine: A Large Language Model Driven Agent Framework for Multimodal AutoML in Protein Engineering shows works well but at GPU costs its own authors flag as the main barrier for resource-constrained labs. Meanwhile evedesign: accessible biosequence design with a unified framework has solved the interface problem — biologists can call Generate, Score, and Transform — but still leaves the multi-objective trade-off to manual specification.

This proposal replaces both knobs with mathematics. Property weights are set automatically by the dispersion of scores across the current candidate set — the entropy weight idea, in which an indicator earns weight by discriminating between candidates — and the step size is set by a nonmonotone line search, an optimizer that permits temporary increases in the loss relative to a forgiving moving-average reference value rather than the last iterate. The two pieces are not bolted together arbitrarily: re-weighting the loss mid-run makes the objective jump discontinuously, which a monotone line search interprets as failure and answers with ever-smaller steps, while a nonmonotone reference absorbs exactly this kind of bounded perturbation. The deliverable is a single-run, zero-exposed-hyperparameter design loop that plugs into the evedesign API surface.

Core Claim

The claim is that dynamic, dispersion-based weighting and nonmonotone step-size control are mutually enabling, and that together they remove the two tuning barriers that accessible biosequence design has not solved. Concretely, at iteration $k$ the composite loss over candidate sequence $x$ is

$$L_k(x) \;=\; \sum_{j \in \mathcal{S}} w_j^{(k)}\, \ell_j(x) \;+\; \mu \sum_{c \in \mathcal{C}} \max\bigl(0,\, t_c - m_c(x)\bigr),$$

where $\ell_j$ are "soft" property scores (expression, activity) whose weights $w_j^{(k)}$ come from the entropy weight method computed over the current batch of candidates, and $m_c$ are "hard" properties (minimum folding stability) that enter as thresholded penalties with a fixed multiplier $\mu$ rather than as weighted terms. Step $x_{k+1} = x_k + \alpha_k d_k$ is accepted against a moving-average reference rather than the last value:

$$L_k(x_{k+1}) \;\le\; R_k + \alpha_k \gamma\, d_k^{\top} \nabla L_k(x_k), \qquad R_k = \eta R_{k-1} + (1-\eta)\, L_{k-1}(x_{k-1}),$$

with $\gamma \in (0,1)$ a fixed contraction constant and $\eta$ the averaging parameter. When weights are recomputed every $T$ accepted steps, the composite jumps by $\Delta_k = |L_{k+1}(x_k) - L_k(x_k)|$; the claim, to be proven, is that under bounded cumulative weight drift the Zhang–Hager-style critical-point guarantee survives with $\Delta_k$ entering as bounded additive perturbation, precisely the quantity the nonmonotone slack is designed to tolerate.

Two documented failure modes of each ingredient are fixed by the other. Pure entropy weighting assigns near-zero weight to any indicator with near-constant scores across candidates — the formulation in The Impact of Two-way FDI on GreenDevelopment Quality of the Logistics Industry– Based on the Threshold Effectof Environmental Regulation — which is correct for uninformative indicators but catastrophic when the near-constant property is a vital stability threshold with little tolerance among viable mutants; the hard/soft split routes those into the penalty term so their weight can never silently vanish. And monotone line searches stall in exactly the ill-conditioned, re-scalarized landscapes that multi-property composites create; Variable Metric Proximal Gradient Method with Diagonal Barzilai-Borwein Stepsize shows scalar steps struggling at condition numbers of $10^5$ while nonmonotone diagonal updates run 10% to 40% faster, and Nonmonotone Line Searches Operate at the Edge of Stability shows nonmonotone acceptance with its hyperparameter-free "noTune" resetting finds flatter solutions without any learning-rate input.

The closest existing work, Optimized Particle Swarm Algorithm for Advanced Bi-Level Dispatch in New Energy Power Systems, does combine entropy-based objective weighting with non-greedy search — but inside an evolutionary swarm metaheuristic for power dispatch, with no gradient-based line search, no convergence theory, and no notion of protecting low-dispersion constraints. Our proposal differs in all three respects, and additionally makes the weighting dynamic and coupled to the acceptance rule rather than computed once, statically, before the search begins.

Research Program

The first work item is the constraint-aware weighting rule. Entropy weights will be computed on rank-transformed rather than raw scores, so a single rogue outlier in a deep mutational scanning assay cannot depress an indicator's entropy and hijack its weight — a documented weakness of the method. Indicators are then classified by a two-question test a biologist can answer without numerics: is this property allowed to trade off (soft), or must it stay above a threshold (hard)? Hard indicators move into the penalty term with a threshold supplied in the same units the biologist already thinks in.

The second item is the optimizer. We build the acceptance test above from the moving-average reference of Accelerated Proximal Gradient Methods for Nonconvex Programming, which retains convergence to critical points in nonconvex settings, combine it with the noTune resetting of Nonmonotone Line Searches Operate at the Edge of Stability so no line-search hyperparameter remains, and use the adaptive contraction factor of Adaptive backtracking line search, which adds no extra function or gradient evaluations. A key design choice sidesteps the stochastic-extension open problem flagged by both that paper and Don't be so Monotone: Relaxing Stochastic Line Search in Over-Parameterized Models: in a surrogate-based design loop, the objective is a deterministic function (forward passes of fixed property scorers), so line-search evaluations are cheap and the decrease conditions hold exactly rather than in expectation. This also avoids the interpolation assumption PoNoS needs, which is exactly what noisy biological assay data violates — a tension the corpus itself identifies. Adaptive Gradient Descent without Descent argues that any line-search evaluation is wasteful, but its descent-free rule is proven only for convex objectives and shows a noisy first 75 epochs on networks; we therefore keep a nonmonotone line search and treat AdGD as an ablation, not a component.

The third item is theory and integration. We will prove the bounded-drift convergence lemma sketched above, and attempt the still-open step-size lower bound for nonmonotone steps larger than $2/L$ in our deterministic-surrogate setting. On the engineering side, the loop ships as a module behind the Generate/Score/Transform interface of evedesign: accessible biosequence design with a unified framework, with the entire configuration surface consisting of the soft/hard property list.

Baselines

The naive reference points are zero-shot scoring without any training, which AutoProteinEngine: A Large Language Model Driven Agent Framework for Multimodal AutoML in Protein Engineering reports at an F1-score of 0.4764 on the Brazzein sweetness task, and fixed-default pipelines of the InstantDL kind, whose authors themselves document that freezing optimizer settings costs peak performance. These establish the floor and show what "accessible but untuned" currently buys.

The strongest existing competitor is the AutoProteinEngine AutoML pipeline, which pairs Tree-structured Parzen Estimator search with asynchronous successive halving over multimodal protein models. It is the strongest because it is the only published system that both removes the tuning burden from the biologist and demonstrably beats human experts: it reports an F1-score of 0.7306 on Brazzein sweetness classification against 0.5709 for manual tuning, and on the STM1221 enzyme-activity regression an R² of 0.6805 (RMSE 0.3488) against 0.5965 for manual tuning. We will run its full pipeline on identical data, identical model families, and a matched total GPU-hour budget, against our method's single training run. Beating it means matching or exceeding these F1 and R² figures while consuming at least five times less compute and exposing zero tuning hyperparameters. On the prediction side, the supervised benchmark to beat is ProteinNPT: Improving Protein Property Prediction and Design with Non-Parametric Transformers, which holds the supervised top spot on ProteinGym: Large-Scale Benchmarks for Protein Fitness Prediction and Design with a Spearman correlation of 0.613; unlike our proposal, ProteinNPT uses fixed, manually combined losses, so it isolates what adaptive weighting adds on standard prediction.

The decisive ablations separate the two ingredients. Uniform weights ($w_j = 1/M$) with our nonmonotone optimizer isolate the weighting contribution; entropy weights with a grid-tuned Adam isolate the optimizer contribution; plain entropy weighting without the hard/soft split should reproduce the zero-weighting pathology on a near-constant stability indicator, demonstrating why the constraint term is necessary rather than decorative.

Why not just…?

Why not just use Adam with a cosine schedule? Three pieces of evidence in the corpus say this falls short. Don't be so Monotone: Relaxing Stochastic Line Search in Over-Parameterized Models shows that finding the right learning rate for Adam or SGD requires expensive grid searches, which is precisely the compute the accessibility gap cannot afford. InstantDL documents that keeping a fixed default schedule sacrifices peak performance across diverse biological datasets. And Variable Metric Proximal Gradient Method with Diagonal Barzilai-Borwein Stepsize shows scalar schedules stalling on ill-conditioned objectives — which a multi-property composite becomes by construction. Beyond speed, a schedule assumes a stationary objective, which re-weighting breaks by design. Our matched-budget comparison against tuned Adam settles the question empirically; if a well-tuned Adam matches us at equal total compute including its tuning cost, the optimizer claim collapses, and we will report that honestly.

Why not just use equal weights or expert-set weights? Uniform weighting spends gradient budget on indicators that fail to discriminate candidates — the core rationale of the entropy method — and manual weighting of the AHP kind reintroduces exactly the expert judgment that accessible platforms exist to remove. The subtler argument for our approach is that informativeness is not static: an indicator that is uninformative across a broad initial population becomes discriminative as optimization narrows the candidates, which static weights of any kind cannot track. That said, this alternative is a real threat on small property sets where all indicators are informative throughout; our uniform-weight ablation is designed to detect exactly the regime where entropy weighting buys nothing.

Why not just wrap an AutoML library in a friendly interface? This has been done: AutoProteinEngine is that system, and its own authors list high GPU requirements as the urgent limitation blocking adoption in small labs. The structural reason is that outer-loop hyperparameter search must train dozens of candidate models as a black box, while nonmonotone line search self-tunes inside a single run at zero additional evaluations, as Adaptive backtracking line search demonstrates with up to 82.7% elapsed-time reduction for Adagrad and 93.5% on nonconvex matrix factorization. We are honest about the threat: AutoProteinEngine also automates data retrieval and offers a conversational interface we do not attempt; our contribution is the optimization core such agents could call. If its full pipeline at matched compute beats our single run by a wide margin on the shared tasks, our claim narrows to compute efficiency and the weighting rule, and the experiments are constructed to reveal which.

Proposed Experiments

The prediction track uses the ProteinGym benchmark suite (217 substitution assays and 66 indel assays, over 2.7 million mutated sequences), training multi-property predictors with the entropy-weighted multitask loss and the nonmonotone optimizer. The headline comparison is Spearman correlation against the reported ProteinNPT supervised top result of 0.613, plus GPU-hours against a TPE/ASHA sweep of the AutoProteinEngine kind on the Brazzein sweetness classification task (its reported F1: 0.7306) and the STM1221 enzyme-activity regression (its reported R²: 0.6805, RMSE 0.3488). Success is defined as matching or exceeding these numbers at a fifth of the compute with no exposed learning rate or weight hyperparameters.

The design track exercises the full loop on multi-objective tasks with a genuine hard constraint: the chorismate mutase and kynureninase datasets used by evedesign for conditional enzyme design, where designed candidates can be scored against measured deep mutational scanning fitness. No reported result was found on Lacuna for either EcCM or KYNase, so we will establish our own reference numbers under equal weights, manual weights, and static entropy weights. Metrics are the mean measured fitness of top-k designs, the constraint-violation rate for the stability threshold, and the number of surrogate evaluations to reach a target fitness, where we expect the nonmonotone acceptance to show its advantage in evaluation counts analogous to the 146-versus-994 iteration gap Accelerated Proximal Gradient Methods for Nonconvex Programming reports on sparse logistic regression.

The diagnostic track makes the mechanism visible. We will trace step sizes and rejection counts at each re-weighting event, predicting that a monotone Armijo rule rejects and shrinks where the nonmonotone reference accepts; we will construct a controlled candidate population in which a critical stability indicator is nearly constant and show that plain entropy weighting zeroes it out while the hard/soft split keeps every design feasible; and we will measure the sharpness of found solutions using the cheap sharpness estimator of Nonmonotone Line Searches Operate at the Edge of Stability, testing whether flatter surrogate minima yield designs more robust to scorer uncertainty.

What would falsify the proposal is stated plainly. If tuned Adam at matched total compute matches our wall-clock and final quality, the optimizer half is unnecessary. If uniform weights match entropy weights across the design tasks, the weighting half is unnecessary. If AutoProteinEngine's sweep at matched GPU-hours beats our single run by more than two points on F1 or R², the accessibility argument reduces to a compute footnote. Each experiment above is built to be able to return exactly that verdict.