Biologically Informed Priors for Sequence and Structure Generation

31 concepts23 papers1 proposal

When designing novel proteins, RNA, or therapeutic peptides, generative models face an astronomical search space. The space of possible amino acid sequences for a modest 100-residue protein contains $20^{100}$ combinations—more than the number of atoms in the observable universe. Within this vast expanse, the configurations that fold into stable, functional 3D structures are vanishingly rare.

Traditional generative frameworks, such as standard diffusion models or flow matching architectures, typically navigate this space by starting from a blank slate: isotropic Gaussian noise or uniform categorical distributions. However, forcing a model to learn the fundamental laws of biophysics and evolutionary grammar from scratch is computationally inefficient and frequently leads to "hallucinated" designs that fail in the wet lab.

To overcome this, a powerful paradigm has emerged: biologically informed priors. Instead of initializing or constraining generative models with uninformative noise, researchers are embedding evolutionary, taxonomic, and structural constraints directly into the generative loop. This approach provides models with a shortcut to "biological common sense," steering them exclusively toward viable regions of the macromolecular landscape.


Shifting the Starting Line: From Noise to Data-Driven Priors

In a standard geometric diffusion or flow matching model, the generation of a 3D molecular structure begins by sampling random coordinates from a Gaussian distribution and iteratively denoising them into a coherent backbone.

Standard Approach:  [Gaussian Noise] --------(Iterative Denoising)--------> [Protein Structure]
Informed Approach:  [pLM Structural Prior] --(Refinement/Flow Matching)--> [Optimized Structure]

The authors of FlowDesign and Alex Morehead et al. (FlowDock) demonstrated that replacing this random initialization with data-driven structural priors significantly improves generation accuracy. For instance, FlowDock utilizes protein-specific structural priors derived from pre-trained protein language models (pLMs) to represent the initial unbound state of a protein-ligand complex rather than relying on generic harmonic priors.

Why it works

Initializing a generative trajectory close to a physically plausible configuration fundamentally alters the optimization problem. When a model starts from pure noise, early generation steps are dominated by resolving massive steric clashes and establishing basic local chemistry (like valid bond lengths and angles). By starting from an informed structural prior, the model's capacity is freed to focus on high-level design objectives—such as optimizing binding interfaces or structural stability—resulting in faster convergence and higher sequence recovery.


Language Models as Evolutionary Guardrails

Another prominent mechanism for injecting biological priors is the use of pre-trained language models as soft constraints during sequence generation or optimization.

When a surrogate model or fitness predictor is used to guide sequence design, it often suffers from exploitation; the optimizer finds "adversarial" sequences that score highly on the noisy predictor but are biologically impossible. To prevent this, models like PROSPERO, developed by Michal Kmicikiewicz et al., pair a surrogate fitness predictor with a frozen, pre-trained generative model. The pre-trained model acts as an evolutionary filter, vetoing sequence suggestions that violate the learned grammar of natural proteins.

A mathematically elegant implementation of this concept is detailed by Sebastian W. Ober et al. in the context of antibody optimization. They modify the acquisition function of a Bayesian Optimization loop by multiplying it directly by the pseudo-likelihood of the sequence under a pre-trained pLM:

$$\alpha_{\text{informed}}(x) = \alpha_{\text{standard}}(x) \cdot P_{\text{pLM}}(x)$$

Why it works

Large language models trained on millions of natural sequences (whether protein or RNA, as explored in EVO-RDesign by Xiaoran Jiao et al.) implicitly capture the complex, non-local epistatic interactions required for folding. The pseudo-likelihood $P_{\text{pLM}}(x)$ penalizes sequences that deviate from natural evolutionary patterns.

This joint sequence-structure awareness is also critical in co-design models like CarbonNovo (Milong Ren et al.), FOLDFLOW-2 (Guillaume Huguet et al.), and AbX (Tian Zhu et al.). By embedding pLM representations directly into the structural generation loop, these frameworks leverage the fact that evolutionary sequence patterns contain implicit blueprints for physical folding requirements, ensuring that generated 3D backbones remain designable and physically plausible.


Conditioning on Evolutionary Context and Lineages

Rather than relying on a global prior for all proteins, generation can be constrained using fine-grained evolutionary context, such as homologous families or taxonomic lineages.

The PoET architecture, proposed by Timothy F. Truong Jr. et al., treats protein families as "sequences-of-sequences." At inference time, the generative model is conditioned on a set of functional homologs retrieved from a database. Similarly, TaxDiff (Li Hao et al.) incorporates the taxonomic hierarchy directly as a conditioning signal in a diffusion model to generate species-specific sequences.

Evolutionary Conditioning:
[Database Homologs] ---\
                        +---> [Conditional Generator] ---> [Novel Functional Sequence]
[Target Specification] -/

Why it works

Natural evolution has already performed the heavy lifting of filtering out non-functional variants within specific protein families or species. By conditioning the generative process on these evolutionary constraints, the model does not need to rediscover the specific structural rules of a given fold. Instead, it interpolates within a pre-verified, biologically viable sub-space, maintaining high structural similarity to natural templates while generating novel sequences.


Grounding Generation in Conserved Structural Motifs

In many design scenarios, the prior is not a statistical distribution but a hard geometric constraint: a specific arrangement of amino acids responsible for a protein's function, such as an enzyme's catalytic triad or an antibody's binding loop.

Frameworks like NAEPro (Zhenqiao Song et al.) and EnzyControl (Chao Song et al.) implement motif-constrained scaffold design. They treat functional motifs as immutable coordinates in 3D space, forcing the generative model to build the surrounding protein scaffold around these fixed anchors.

Taking this a step further, PRISM (Sazan Mahbub et al.) introduces a multimodal Retrieval-Augmented Generation (RAG) framework that explicitly retrieves fine-grained structural motifs from a database of known proteins to ground inverse folding predictions.

Why it works

Instead of forcing a neural network to perfectly memorize and reconstruct delicate local chemistry purely from its weights, motif-grounding provides an explicit inductive bias. It guarantees the geometric integrity of the active site from step zero. The model can then stochastically explore diverse scaffold architectures that support that specific geometry, allowing for the creation of highly divergent, novel sequences that nevertheless preserve the exact target biological activity.

By combining these diverse strategies—structural initializations, language model guardrails, evolutionary conditioning, and geometric motifs—modern generative models move away from unconstrained hallucination and toward highly efficient, biologically viable macromolecular design.

Key Foundational Work

Functional Geometry Guided Protein Sequence and Backbone Structure Co-Design (2023) — contributed 2 concept elements to this direction.
PoET: A generative model of protein families as sequences-of-sequences (2023) — contributed 1 concept elements to this direction.
Bootstrapped Training of Score-Conditioned Generator for Offline Design of Biological Sequences (2023) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_biologically_informed_reference_process_b67dc059,
  title = {biologically informed reference process},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/biologically-informed-reference-process/rdr_b67dc059c3454494ad14211163491c80},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_biologically_informed_reference_process_b67dc059_v1,
  title = {biologically informed reference process},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/biologically-informed-reference-process/rdr_b67dc059c3454494ad14211163491c80/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake