Debiased Discrete Diffusion for Narrow-Spectrum Livestock Antimicrobials

Generative models do not copy their training data; they magnify its skews. A generator trained on a dataset in which 90% of examples share one attribute will produce that attribute even more often than the data does, a failure mode called bias amplification or majority-mode collapse. This is well documented for faces and text — a conditional GAN trained on CelebA under-produces the already-rare "blond male" combination, generating it in 3.97% of samples against a true 5.85% prevalence FICS, Nam et al. — but it has never been measured in the domain where the skew has direct clinical consequences: biological sequence generation. Public antimicrobial peptide databases are overwhelmingly dominated by one molecular recipe: short, strongly positively charged, amphipathic alpha-helices that tear open the membranes of broad classes of bacteria. The peptides that actually work as selective oral therapeutics in livestock are the opposite profile — low-net-charge, protease-stable, receptor-targeted molecules such as the lasso peptide microcin Y, which lifted chick survival from 83.33% to 96.67% under Salmonella Pullorum challenge while expanding beneficial commensals like Barnesiella Microcin Y in chicks. These narrow-spectrum molecules are a tiny minority mode of the training data that a biased generator will systematically under-produce.

This proposal does two things. First, it audits whether state-of-the-art peptide generators amplify the majority "cationic helix" mode — there is already a hint: a latent diffusion model over protein-language-model embeddings was observed to over-produce lysine relative to natural peptides AMP-Diffusion. Second, it ports the strongest known debiasing recipe — time-dependent importance reweighting with an explicit score-correction term, developed for continuous image diffusion TIW-DSM, Kim et al. — into discrete token diffusion for peptides, using a small reference corpus of narrow-spectrum microcins and colicins. The endpoint is not a classifier score but a wet-lab selectivity measurement: peptides that kill livestock pathogens (Salmonella Pullorum, enterotoxigenic E. coli W25K) while sparing the commensal gut flora whose preservation is the entire point of replacing antibiotic growth promoters.

Core Claim

The claim has three parts. First, majority-mode amplification is real, measurable, and unaddressed in discrete peptide generation: today's best model conditions on a scalar "antimicrobial probability" from the Macrel predictor, which by construction steers sampling toward the densest, most conventional region of AMP sequence space AMP-DiT. We can quantify this with the same amplification statistic used in the vision literature: for an attribute $A$, the shift $\Delta_A = q_{\text{gen}}(A) - q_{\text{data}}(A)$, where $q_{\text{gen}}$ and $q_{\text{data}}$ are the attribute frequencies in generated versus training sequences FICS, Nam et al.. Second, counteracting it requires more than reweighting the training loss: the continuous-diffusion ablation showed that loss weights alone cannot pull the reverse trajectory out of majority modes, and that an active correction to the learned vector field is mathematically necessary TIW-DSM, Kim et al.. Third, the minority mode we aim to recover — narrow-spectrum, commensal-sparing peptides — is exactly the mode with validated in vivo efficacy in livestock Microcin Y in chicks; Colicins in swine, so debiasing is not an aesthetic fix but a direct route to the molecules the application needs.

The biology side supplies the constraint and the evaluation regime, not the mechanism. Replacing antibiotic growth promoters demands agents that suppress enteric pathogens without the collateral destruction of commensal microbiota, because that destruction is what triggers the inflammation and dysbiosis the antibiotics were masking Ding et al.. Broad-spectrum cationic helices — the majority mode — carpet-bomb precisely the Lactobacillus and Barnesiella populations that probiotic and microcin interventions are observed to expand Microcin Y in chicks; Ma et al.. No generative peptide model has ever been evaluated on this axis: no reported result found on Lacuna for commensal-sparing selectivity of AMP-DiT, AMP-Diffusion, or any generative antimicrobial peptide model.

Research Program

The first workstream is an audit. We will train two generators on the same skewed corpora — the 195,121-sequence peptide database assembled by the latent-diffusion work AMP-Diffusion and the 82,359-sequence ESCAPE benchmark, whose 21,409 antimicrobial sequences carry five functional labels and whose rare classes are already known to suffer severe performance drops even in discriminative classifiers (a 35.7% relative average-precision gain was needed just to recover the rare antiparasitic class) ESCAPE benchmark. For attributes such as lysine fraction, net charge, predicted helical-amphipathic class, and microcin/colicin-likeness, we measure $\Delta_A$ plus distributional divergences (total variation, Jensen-Shannon, $\chi^2$) between generated and training attribute distributions, following the evaluation protocol of the optimal-control alignment work Luan et al.. This audit is not a formality: it is the decisive test of whether bias amplification, established for pixels and text Rosenberg et al.; Fan et al.; Ducel et al., also afflicts discrete sequence diffusion. If it does not, we will report that and stop.

The second workstream is the method, which we call discrete time-dependent importance reweighting (dTIW). The backbone is a discrete diffusion transformer — a denoising model that learns to reverse a categorical noising process over amino-acid tokens, the architecture validated by AMP-DiT — trained on the full skewed database. We then train a small time-dependent discriminator on noised token sequences to distinguish database samples from a reference corpus of narrow-spectrum peptides. The reference corpus is built from microcin, colicin, and bacteriocin entries in DBAASP and the mining repositories used by AI-Driven Antimicrobial Peptide Discovery, plus the complete genomes of the commensal E. coli Y18J and the pathogen ETEC W25K Colicins in swine — likely a few hundred sequences, which is exactly the small-reference regime TIW-DSM was designed for. The key trick we inherit is that noise bridges the density chasm: at high noise levels the two corpora overlap heavily in token space, so the density ratio is estimable where it would be astronomically extreme at $t=0$ TIW-DSM, Kim et al.. The ratio enters twice. As a loss weight, it emphasizes reference-aligned training sequences at each noise level. As a score correction — the ingredient the continuous ablation proved indispensable — it tilts the model's per-step prediction of the clean sequence:

$$\tilde{p}_\theta(x_0 = v \mid x_t, t) \;\propto\; p_\theta(v \mid x_t, t)\cdot \exp\!\big(\lambda_t\, g_\phi(x_t, v, t)\big),$$

where $p_\theta(v \mid x_t, t)$ is the denoiser's predicted probability that position $t$'s corrupted token should be restored to amino acid $v$, $g_\phi$ is the token-level discriminator logit favoring reference-corpus tokens, and $\lambda_t$ is a noise-level-dependent strength. This is the discrete analogue of bending the score field away from majority modes: instead of nudging a continuous vector field, we multiplicatively adjust the categorical distribution the denoiser predicts at every reverse step. No retraining of the backbone is needed for the correction, which lets us sweep $\lambda_t$ cheaply and, importantly, lets us apply the correction only at inference time as an ablation.

The third workstream is evaluation, moving down a ladder of biological fidelity. In silico, we track the standard generation metrics reported by the strongest baseline — antimicrobial rate, non-hemolytic safety score, OmegaFold mean pLDDT, diversity AMP-DiT — so that debiasing is never allowed to trade away basic quality. In vitro, we run minimum inhibitory concentration panels against Salmonella Pullorum and ETEC W25K on one side and commensal Lactobacillus reuteri LR6 (the stress-tolerant isolate from the 52-strain screen of Ma et al.), Bifidobacterium, and Barnesiella-relevant anaerobes on the other, defining a selectivity index as commensal MIC divided by pathogen MIC. We also challenge candidates with pepsin, trypsin, and chymotrypsin, because the in vivo success of microcin Y was explicitly attributed to its lasso fold resisting proteolysis Microcin Y in chicks — a stability axis no generative peptide model currently optimizes. Finally, the top two or three candidates go into the specific-pathogen-free chick Salmonella Pullorum model at 20 mg/kg oral gavage, with survival, cecal pathogen load, tight-junction transcripts (ZO-1, CLDN-1), and microbiota shift as endpoints, benchmarked directly against the microcin Y numbers.

Baselines

The naive reference point is unconditional generation followed by post-hoc filtering: sample a large pool from an unguided generator, then keep whatever passes an activity classifier and a hemolysis predictor. This is how the earlier generative peptide systems operated, and the strongest model in the field documents how badly it fails — AMPGAN, PepCVAE, HydrAMP, and AMP-Diffusion achieved antimicrobial rates of only 0.12 to 0.69, versus 1.00 for classifier-guided conditional diffusion AMP-DiT. We will run this baseline in its modern form, on our exact data split and compute budget, and we will strengthen it into its best possible version by adding FICS-style corrective rejection sampling — filtering generated batches to exact target attribute proportions using a discriminator, which lifted minority generation from 3.97% to 6.13% (and to 47.11% under a semi-supervised target) in the vision setting FICS, Nam et al.. If sampling-and-filtering can hit our target narrow-spectrum mode share at acceptable quality, the added training machinery is not justified, and this baseline is where that gets decided.

The strongest existing competitor is AMP-DiT itself: a discrete diffusion transformer that injects continuous Macrel-derived activity and safety scores into its normalization layers, achieving an antimicrobial rate of 1.00 on external predictors, a 0.94 non-hemolytic safety score, mean OmegaFold pLDDT of 75.25, and the highest diversity score (47) among its baselines AMP-DiT. It is the strongest competitor because it already solved discrete representation and conditional guidance, and because its scalar-guidance recipe is the obvious cheap way to chase any property. We will retrain it on our corpora with the same token budget, the same judge models (Macrel, AMP-Scanner, AMPlify), and the same tuning budget (a fixed random-search allocation shared across all methods), so the comparison isolates debiasing rather than compute. Beating it means, at matched or better antimicrobial rate, safety, pLDDT, and diversity, achieving a substantially larger share of generations in the narrow-spectrum reference mode (measured as lower Jensen-Shannon divergence to the reference corpus's attribute distribution) and a higher in vitro selectivity index. AMP-DiT reports no number on either axis — no reported result found on Lacuna for narrow-spectrum or commensal-sparing metrics from AMP-DiT — so this comparison will produce the first such measurements regardless of outcome.

The key ablation is dTIW with reweighting only, the correction term $\lambda_t g_\phi$ switched off. Its continuous-space counterpart failed: time-independent reweighting scored an FID of 15.79 on CIFAR-10 long-tail, worse than the 12.99 of naive unweighted training, while the full method with score correction reached 11.51, and an explicit ablation showed weights alone cannot escape majority modes TIW-DSM, Kim et al.. If the discrete W-only variant matches the full method on mode share and downstream selectivity, then the transfer of the correction term — our central technical claim — is unnecessary, and we will say so. Secondary ablations isolate the reference corpus (leave-one-family-out, to address the known fragility of reference-set methods to artifacts in $D_{ref}$) and the discriminator granularity (sequence-level versus token-level).

Why not just…?

Why not just generate a million peptides and filter them? Because the yield collapse is documented at the easy end of the problem and gets worse at the hard end. Filtering-based generators produced antimicrobial rates of 0.12 to 0.69 versus 1.00 for guided generation AMP-DiT, and that is for "antimicrobial at all," a property with cheap, well-calibrated classifiers. Our target property — kills one pathogen, spares commensals, survives proteases — has no reliable in silico predictor at all; the only ground truth is MIC assays, so post-hoc screening pays a wet-lab cost per candidate. The vision literature adds a subtler failure: naive filtering inherits the generator's under-sampling of the minority mode rather than correcting it FICS, Nam et al.. We take this alternative seriously enough to run its strongest form — corrective rejection sampling to exact target proportions — as an explicit baseline, so the question is settled empirically rather than by argument.

Why not just resample or statically reweight the training data? This is the most tempting shortcut: upweight the few hundred microcin and colicin sequences and retrain. The continuous-diffusion evidence says it fails for a structural reason — the density chasm. With a reference corpus hundreds of times smaller than the database and nearly disjoint in sequence space, the density ratio at low noise is extreme and unestimable; static, time-independent weights computed once propagated error across the whole denoising trajectory and scored worse than doing nothing (FID 15.79 versus 12.99 naive, against 11.51 for the time-dependent method) TIW-DSM, Kim et al.. Our W-only ablation is precisely this alternative implemented as well as we can implement it, so if simple resampling suffices in discrete space, the ablation will reveal it — and we would then report the simpler recipe.

Why not just use the natural peptides that already work? Microcin Y and the Y18J colicins are validated, so why generate anything? Three reasons from the literature. The natural supply is thin and breed-bound: colicin B and M genes are enriched in indigenous Laiwu pigs and scarce in commercial Duroc and boar herds Colicins in swine, and finding one resilient probiotic strain required screening 52 candidates Ma et al.. Resistance to narrow-spectrum agents arises through receptor mutation, so a single natural molecule is a single point of failure; a generator that reliably produces the narrow-spectrum mode yields a pipeline of structural variants as resistance hedging. And natural mining cannot explore the combinatorial space beyond what evolution happened to sample, which is the entire motivation for the generative line of work AMP-DiT; AI-Driven Antimicrobial Peptide Discovery. We are honest about the bar: microcin Y already achieves 92.7% to 99.4% cecal pathogen reduction and 96.67% survival Microcin Y in chicks, so our candidates must match that in vivo while adding diversity the natural molecule lacks.

Proposed Experiments and Decisive Checks

Experiment one is the audit: train the latent and discrete generators on the 195,121-sequence corpus AMP-Diffusion and ESCAPE ESCAPE benchmark, sample 50,000 sequences from each (matching the sampling protocol of AMP-Diffusion), and report $\Delta_A$ with confidence intervals for lysine fraction, net charge, and microcin-likeness. The check is strict: if the attribute shift is not significantly worse than the database's own skew — that is, if discrete diffusion amplifies nothing — the premise fails and the project stops at a publishable negative audit.

Experiment two is the method head-to-head: dTIW versus AMP-DiT, versus filtering with corrective rejection sampling, versus W-only dTIW, all on identical data, compute, and tuning budgets. Primary metrics are mode share of the narrow-spectrum reference class and Jensen-Shannon divergence of the generated attribute distribution to the reference corpus; quality gates are the AMP-DiT-reported scales — antimicrobial rate against its 1.00, safety against its 0.94, pLDDT against its 75.25, diversity against its 47 AMP-DiT. Calibration for what a good generator delivers in the lab comes from the Riemannian-navigation work, which achieved a 100% validation rate at $\le 32\ \mu\text{g/mL}$ and 82% potency at $\le 4\ \mu\text{g/mL}$ across 29 synthesized peptides on a 19-strain panel including 8 multidrug-resistant clinical isolases PepCompass — noting that PepCompass tested only human hospital pathogens and older VAE backbones, so our livestock pathogen-plus-commensal panel is a strictly harder selectivity question it never attempted.

Experiment three is the biological ladder: synthesize roughly thirty candidates (the PepCompass cohort size), run MIC panels against Salmonella Pullorum and ETEC W25K versus the commensal panel, with protease challenge, and take the top performers into the chick challenge trial with the microcin Y protocol as the reference standard. The decisive check for the whole proposal is a single comparison: at matched in vitro budget, does dTIW deliver a higher fraction of candidates with pathogen MIC $\le 32\ \mu\text{g/mL}$ and a selectivity index above, say, 8, than AMP-DiT and than filtering? If it does, debiased discrete diffusion is a practical route to post-antibiotic livestock therapeutics; if it does not, the honest conclusion is that scalar guidance plus screening suffices for this application, and bias amplification in sequence space, while real, is not the binding constraint.