Generative models that design proteins inherit the skews of the databases they are trained on, and those skews quietly decide which proteins get designed. Protein language models — transformers trained on natural sequences to score and generate new ones — systematically favor sequences from well-sampled organisms like E. coli and humans: species identity alone explains up to 50% of the variance in the likelihood scores that practitioners use as a proxy for fitness, and when these models are asked to design proteins for hot-salt or high-temperature microbes, 63% of the resulting designs lose thermostability and 83% gain the wrong charge for high-salinity environments (Protein language models are biased by unequal sequence sampling across the tree of life). Antibody models hallucinate naive templates back onto mutated sequences, predicting germline residues 85–95% of the time exactly where binding affinity lives (Addressing the antibody germline bias and its effect on language models for improved antibody design). And the neural structure predictors used to verify designs have their own blind spots, which generation methods learn to exploit: candidates that score near-perfectly in silico and then fail completely in wet-lab binding assays (ProtDBench: A Unified Benchmark of Protein Binder Design and Evaluation).
The machine learning fairness literature spent a decade learning that this kind of bias is a pipeline pathology, not a point defect. Balancing the data does not stop a model from reconstructing the bias through correlated proxy features; recalibrating outputs after the fact degrades utility when the internal representations are already corrupted; and the Pareto frontier between task performance and equity that matters is only reachable by stacking interventions across data, training, and inference (Jaeyoung Park et al.; Jiaee Cheong). Protein design has, so far, only ever fixed one stage at a time. This proposal takes the fairness lens seriously as an engineering discipline: build the first end-to-end stacked debiasing pipeline for functional protein design, and add one genuinely new mechanism that the fairness literature has not needed — using disagreement between independent structure verifiers as a detector and penalty for adversarial shortcuts during generation.
Core Claim
Bias in functional protein design compounds across the pipeline — from skewed training databases, through loss functions that reward majority templates, to verifier filters whose artifacts generators learn to exploit — and therefore no single-stage fix can recover the functional sequences that are currently being lost. I claim three things. First, that a stacked pipeline combining phylogenetic stratification at the data stage, reweighted and invariance-regularized objectives during training, and depth-calibrated multi-verifier filtering at inference, Pareto-dominates every existing single-stage intervention on wet-lab-anchored benchmarks. Second, that test-time steering against a single neural oracle — the currently fashionable approach — measurably increases traversal of the "adversarial ridge" documented by ProtDBench, and that penalizing cross-verifier disagreement during steering collapses this failure mode. Third, that the stages are not independent: pre-processing stratification without in-processing regularization fails (as fairness theory predicts), and post-processing recalibration cannot recover signal that training has already collapsed.
What Prior Work Fixes, One Stage at a Time
Each documented bias in protein design has received exactly one intervention, at exactly one stage, and each paper openly flags the missing stages. Ding & Steinhardt audit species bias at the data level and suggest down-sampling over-represented clades, but implement nothing and leave open whether pruning harms general biochemical grammar (Protein language models are biased...). AbLang-2 fixes germline bias in-processing with a focal loss that down-weights easy residues, cutting non-germline perplexity from roughly 25–30 to 10–12, but touches neither the data distribution nor inference, and leaves open whether forcing non-germline predictions destabilizes the folded scaffold outside the binding loops (AbLang-2). Ripoll et al. fix the mutational-depth score shift post-hoc, converting raw model scores into percentile ranks against a random-mutation background at each depth, which improves recovery of functional distant variants across 76 proteins and 14 deep mutational scanning datasets — but the biased representations inside the model remain untouched (Removing bias in sequence models of protein fitness). ProVADA steers at test time with directed annealing against a black-box localization oracle, reporting a 9.5-fold fitness gain, yet if that oracle carries taxon-specific biases the steering amplifies them (ProVADA: Generating Subcellular Protein Variants via Ensemble-Guided Test-Time Steering). DynamicMPNN fixes the static-snapshot bias of structural databases through joint training on conformational ensembles — a 31% reduction in decoy-normalized RMSD over multi-state ProteinMPNN — but relies on a single AlphaFold3-style oracle for evaluation, inheriting that oracle's artifacts (Multi-state Protein Sequence Design with DynamicMPNN). ProtDBench diagnoses verifier bias but proposes no mitigation (ProtDBench).
The fairness literature supplies both the method and the warnings. Park et al. and Cheong showed empirically that stacking pre-, in-, and post-processing reaches utility–fairness frontiers unreachable by any single stage. Fair Diffusion showed how bias compounds through a pipeline — a 70% male skew in training images amplifies to 90% in generation through the text encoder — and simultaneously showed that deployment-stage steering of intermediate latents can pull outputs back to a 50% ± 4% equitable band, which is the closest existing analogue to my disagreement-penalized steering (Auditing and instructing text-to-image generation models on fairness). Two negative results constrain the design: Lu Zhang et al. proved that perfectly balanced data still leaks bias through proxies, so pre-processing alone is insufficient; and Wohlrapp et al. showed that upstream mitigation fails to propagate reliably through multi-stage pipelines in clinical imaging, so the stages must be co-designed rather than bolted together (Evaluating the Impact of Medical Image Reconstruction on Downstream AI Fairness and Performance). No work in either literature has built or evaluated a stacked pipeline for protein generative models, and none has used cross-verifier disagreement as a bias signal.
Research Program
The pipeline has three coordinated stages, each targeting a bias that is already quantified in the literature, plus one new diagnostic instrument.
At the data stage, the intervention is stratified pairing rather than pruning. Naive down-sampling of over-represented clades risks destroying the sequence grammar that stable folding depends on — the concern Ding & Steinhardt leave open, and which the antibody case makes concrete, since naive germline sequences carry the folding scaffold (AbLang-2). Instead, I will construct counterfactual training pairs in the spirit of the LLM-based augmentation of Zheng et al. (Bot Meets Shortcut: How Can LLMs Aid in Handling Unknown Invariance OOD Scenarios?): ortholog pairs matched for function but differing in taxonomic origin, and germline–hypermutated antibody pairs differing only in somatic mutations. The training signal then comes from the difference between paired sequences, which cannot be explained by species identity.
At the training stage, two objectives. A focal-style loss — following AbLang-2's demonstration that down-weighting easily predicted residues redirects capacity toward rare functional mutations — applied to residues in functional motifs and non-germline positions. And a contrastive invariance objective that pushes the model's likelihoods to be stable across the paired counterfactuals, directly decoupling phylogenetic origin from predicted fitness. DynamicMPNN's result that joint learning across conformational ensembles beats post-hoc logit averaging by 31% RMSD justifies extending the same joint training over the paired set rather than averaging single-model scores (DynamicMPNN).
At the inference stage, two calibrations. First, Ripoll-style percentile recalibration against depth-matched backgrounds, which is cheap and composable (Ripoll et al.). Second — the new mechanism — disagreement-penalized multi-verifier steering. Let $s_v(x)$ be the score verifier $v$ (AlphaFold2, Protenix, Boltz-1) assigns candidate $x$, and $\bar{s}(x)$ their mean. During any test-time search (ProVADA-style annealing, or hallucination as in MotifCraft, MotifCraft: scalable functional protein binder design with AlphaFold2 hallucination), replace the single-oracle objective with
$$S(x) \;=\; \frac{1}{|V|}\sum_{v \in V} s_v(x) \;-\; \lambda \sqrt{\tfrac{1}{|V|}\sum_{v \in V}\bigl(s_v(x) - \bar{s}(x)\bigr)^2},$$
where $\lambda$ controls how strongly cross-verifier disagreement is penalized. The intuition is simple: a candidate that genuinely binds should convince independently trained verifiers, while a candidate exploiting one model's mathematical blind spots will not. ProtDBench showed these verifiers capture complementary regions of sequence space — AF2-approved binders are frequently rejected by Boltz-1 and vice versa — so their disagreement is a usable signal for shortcut detection (ProtDBench). For generation backbones I will use an open SE(3) flow-matching model (microsoft/protein-frame-flow) alongside ProteinMPNN and ESM2 priors, keeping everything reproducible.
Proposed Experiments
The program runs five studies, each anchored to a benchmark with a reported number to beat or explain.
The first study is diagnostic and settles the open tension between ProVADA and ProtDBench. I will rerun ProVADA-style directed annealing and MotifCraft-style hallucination against a single verifier on the ProtDBench 10 targets, and measure the disagreement rate — the fraction of candidates that pass one verifier and fail another — against the wet-lab ground truth of the Cao dataset (478,000 non-binders, 1,385 confirmed binders, as reported by ProtDBench). The hypothesis is that steering gain and disagreement rate rise together, i.e., that part of the 9.5-fold gain reported by ProVADA is adversarial-ridge traversal rather than genuine function (ProVADA). This is falsifiable: if disagreement is flat or falls during steering, the multi-verifier penalty loses its motivation and the proposal narrows to the stacked pre/in-processing claims.
The second study evaluates the full stacked pipeline on variant recovery: the 76 proteins and 14 deep mutational scanning datasets used by Ripoll et al. (GFP, PhoQ, GCN4, AAV2 capsids), with the metric being top-1% recovery of functional variants at mutational depths $n = 1, 5, 10$, plus zero-shot transfer to ppluGFP at 18% sequence identity (Ripoll et al.). The comparison isolates stacking: percentile recalibration applied to a biased model (Ripoll's setting) versus applied to a model trained on paired, invariance-regularized data (mine).
The third study reproduces the extremophile setup of Ding & Steinhardt on the curated 7,545-sequence, 133-species, 203-protein-type benchmark: metrics are thermostability retention for thermophile-targeted designs (baseline: 63% of designs lose thermostability), charge drift for halophiles (baseline: 83% show damaging pI increases), and species-explained variance in likelihood (baseline: up to 50%) (Ding & Steinhardt). Success means cutting all three substantially while preserving general grammar, measured by perplexity on held-out structural benchmarks — the Tension 1 question answered empirically rather than argued.
The fourth study is the antibody track on the curated Observed Antibody Space subset of 1.26 million paired chains: non-germline perplexity (AbLang-2 reports roughly 10–12, down from 25–30) plus predicted structural stability outside the CDR loops, the risk AbLang-2 explicitly leaves unaddressed (AbLang-2). The stacked pipeline should match AbLang-2's perplexity while improving scaffold stability, because the counterfactual pairing teaches the model which residues may mutate and which must not.
The fifth study is the binder track on the ProtDBench 10 targets, using its throughput-aware yield metric $N_{24h}$ on a single NVIDIA A100 and FoldSeek cluster-level diversity, against the seven benchmarked pipelines (RFdiffusion, BindCraft, BoltzGen, PXDesign and others), where ProtDBench reports PXDesign achieving hundreds of successful structures per day versus fewer than 50 for hallucination methods (ProtDBench). The decisive comparison is disagreement-penalized steering versus single-verifier steering under identical compute, scored against the Cao wet-lab labels. Where no reported number exists — for example, MotifCraft's success rate on its 24-motif benchmark and the Adaptyv Bio Nipah competition — no reported result found on Lacuna for those figures, so those resources serve as secondary qualitative checks rather than headline metrics (MotifCraft).
Baselines
The "do nothing" reference point is standard practice: raw ESM2 or ProteinMPNN likelihood ranking with a single-verifier filter, the exact configuration that Ding & Steinhardt showed is confounded by species identity, and the configuration underlying most published design pipelines. It will be run on the same targets and candidate budgets as the stacked pipeline, and any pipeline that does not beat it decisively on functional recovery has no reason to exist.
The strongest single-stage competitors are, per track: Ripoll's percentile recalibration for fitness scoring, which is the reigning post-processing method and the most direct competitor on the DMS benchmark — it achieved qualitative improvement across all 14 DMS datasets in its paper, though no single headline accuracy figure is reported on Lacuna, so I will reimplement it and establish the number myself under matched conditions (Ripoll et al.); AbLang-2's focal loss for the antibody track, with its reported non-germline perplexity of roughly 10–12 as the number to match while improving scaffold stability (AbLang-2); and ProVADA's MADA test-time steering as the strongest competitor for functional yield, which reports a 9.5-fold increase in predicted cytosolic localization fitness over baseline rejection sampling (ProVADA). All baselines will be run on identical data, identical candidate counts, the same single-A100 compute budget using the ProtDBench $N_{24h}$ convention, and the same tuning budget — an equal number of hyperparameter trials per method, swept by the same search procedure, so that no method wins by tuning effort. Beating ProVADA means exceeding its localization-fitness gain while cutting the verifier-disagreement rate and preserving catalytic-dyad geometry, its own secondary metric; beating Ripoll means higher top-1% functional recovery at every mutational depth on the same 14 datasets; beating AbLang-2 means matching its perplexity at better scaffold stability.
The key ablation isolates the proposal's newest ingredient: the full stacked pipeline with single-verifier steering (no disagreement penalty) versus the identical pipeline with the penalty. Any gain attributable to stacking alone must be separated from gain attributable to the multi-verifier term, and the prediction is specific — the penalty should matter most exactly where ProtDBench found the adversarial ridge, in hallucination-style methods that spend over 89% of their compute optimizing against the verifier's loss (ProtDBench). A second ablation removes the pre-processing pairing while keeping the in-processing losses, to test whether the contrastive objective needs counterfactual data or can be built from natural pairs alone.
Why not just…?
Why not just recalibrate the scores after the fact? This is the cheapest option and Ripoll showed it works, but the fairness literature and Ripoll's own limitations point the same way: post-processing ranks candidates against a background distribution, and it cannot resurrect signal that training collapsed. Lu Zhang et al. proved that balanced data still leaks bias through proxies, and Fair Diffusion showed output-level correction cannot undo representational entanglement — a 70% training skew amplified to 90% inside the encoder (Auditing and instructing text-to-image generation models on fairness). Experiment two is designed to settle this directly: percentile recalibration on a biased model versus the same recalibration on a counterfactually trained one. If the gap is negligible, the reviewer is right and the pre/in-processing stages are not worth their cost.
Why not just down-sample the databases? Ding & Steinhardt suggest exactly this, and it is simpler than building paired counterfactuals. But naive pruning is the one intervention the file gives grounds to fear: naive germline sequences carry the folding grammar in antibodies, so starving the model of majority templates risks breaking stability to fix representation (AbLang-2), and Lu Zhang et al.'s proxy result says balance alone does not stop shortcut learning anyway. Experiment three includes a down-sampling arm precisely to test this: if stratified pruning matches paired counterfactual training on thermostability retention at equal grammar cost, I will adopt it and drop the pairing machinery. The honest position is that the file contains no head-to-head evidence either way, which is itself part of what the proposal contributes.
Why not just steer harder at test time against a strong oracle? This is the real threat. ProVADA's 9.5-fold gain is a large number obtained without retraining anything, and MotifCraft shows hallucination scales to real binder problems (ProVADA; MotifCraft). But ProtDBench's adversarial-ridge finding is a direct contradiction: methods that optimize against a verifier's loss exploit its non-physical loopholes and produce in-silico-only success, with wet-lab failure measured against 478,000 non-binders (ProtDBench). Whether ProVADA-style gains are real function or ridge traversal is an open empirical question that no cited paper answers, and diagnostic experiment one is built to answer it with wet-lab labels. If disagreement rates stay flat under steering, then test-time steering against a single oracle is vindicated and my multi-verifier penalty is unnecessary — I would then narrow the proposal to the representation-side claims, which stand independently.
Decisive Checks
Three outcomes would falsify the core claim. If the stacked pipeline does not Pareto-dominate the best single-stage method on any track (DMS recovery, extremophile retention, antibody perplexity-plus-stability, binder yield), the stacking thesis fails and the right conclusion is that protein design biases, unlike fairness biases, do not compound. If disagreement-penalized steering does not reduce wet-lab failure rates relative to single-verifier steering at equal compute, the multi-verifier mechanism fails and the adversarial ridge must be handled some other way. And if pre-processing pairing degrades structural grammar benchmarks rather than improving them, Tension 1 resolves against data intervention and the pipeline should shed its first stage. Each check is a pre-registered comparison on a benchmark with a published baseline number, so none of these questions stays a matter of taste.
References
- Protein language models are biased by unequal sequence sampling across the tree of life
- Addressing the antibody germline bias and its effect on language models for improved antibody design
- ProtDBench: A Unified Benchmark of Protein Binder Design and Evaluation
- Removing bias in sequence models of protein fitness
- ProVADA: Generating Subcellular Protein Variants via Ensemble-Guided Test-Time Steering
- Multi-state Protein Sequence Design with DynamicMPNN
- Auditing and instructing text-to-image generation models on fairness
- Evaluating the Impact of Medical Image Reconstruction on Downstream AI Fairness and Performance
- Bot Meets Shortcut: How Can LLMs Aid in Handling Unknown Invariance OOD Scenarios?
- MotifCraft: scalable functional protein binder design with AlphaFold2 hallucination