Abstract
De novo protein sequence generation models produce valid protein candidates on demand, but are currently either unable to specify what kinds of proteins to generate, or are limited to several broad classes. We present a novel method for controllable de novo protein sequence generation that leverages reference samples to guide generation specified through Gene Ontology (GO) terms. We design an evaluation pipeline based on a Top-TMScore target metric, that prioritises closest-shape matching. We evaluate the effectiveness of our reference-guided controllability approach for protein design across various models, demonstrating high accuracy results when compared with the target benchmarks.
Introduction
In the field of synthetic biology, designing custom proteins is a critical goal for medicine and material science. However, current AI models for protein design face a significant "controllability" gap. Existing tools are often either unconditional—generating valid proteins without functional specificity—or narrowly specialized, limited to the specific types of proteins present in their training data. To design a protein with a specific combination of functions, such as a membrane-bound kinase, researchers typically must undergo expensive retraining or fine-tuning of large models. PRO-GO addresses this by creating a flexible, on-demand generation system that interprets functional requirements specified in natural language.
The PRO-GO Framework
The PRO-GO framework treats protein design as a "few-shot" translation task, leveraging the in-context learning (ICL) capabilities of large language models (LLMs).
Generation Process
- Input Preparation: Users provide target Gene Ontology (GO) terms—standardized descriptors of protein function—and a small set of Reference Sequences (1–5 known proteins) that already exhibit those functions.
- Sequence Generation: These inputs are fed into an LLM (such as Llama 3.1, ProLLaMA, ProtGPT2, or Progen2) with specific instructions. The model uses ICL to recognize functional patterns and generate a novel candidate sequence ($S_n$).
Evaluation Pipeline
A standout feature of the PRO-GO framework is its robust evaluation pipeline, which verifies the structural viability of generated sequences.
The pipeline follows these steps:
- Structure Prediction: The 3D structure of the generated sequence is predicted using ESMFold.
- Structural Comparison: This predicted structure ($S_n'$) is compared against "ground truth" proteins from the Protein Data Bank (PDB) using FoldSeek.
- Scoring: A TM-score ($T_n$) is calculated. A score $\ge$ 0.5 indicates the same fold, while $\ge$ 0.8 indicates very high structural similarity.
Datasets
The study utilized two primary datasets to ensure diversity and accuracy:
- UniRef50: Used to source reference sequences. Clustering proteins at 50% sequence identity ensures the model is exposed to a diverse range of examples.
- Protein Data Bank (PDB): Provided high-quality, experimentally determined structures to serve as the "ground truth" for the evaluation and benchmarking phase.
Key Results and Insights
The researchers found that general-purpose LLMs, specifically Llama 3.1 70B, significantly outperformed specialized protein models like Progen2 when guided by reference samples. This suggests that the reasoning and pattern-matching capabilities of massive LLMs are highly effective for biological sequence design.
- Performance Benchmarks: Llama 3.1 70B achieved a TM-score $\ge$ 0.8 for 91.4% of its generated sequences, nearly matching the 92% score of real proteins in the target benchmark.
- Optimal Context: Providing five reference sequences was identified as the "sweet spot," offering sufficient context without exceeding model limits or introducing noise.
- Scaling Effects: A significant performance gap was observed between the 8B and 70B versions of Llama 3.1. The 8B model only reached the 0.8 threshold 55.7% of the time, emphasizing the importance of model scale.
- Zero-Shot Limitations: Models without reference-guided steering failed to maintain accuracy as structural requirements became more stringent.
Limitations and Future Directions
While high TM-scores indicate correct protein folding, they serve as a proxy for functional activity. Future work will involve "wet-lab" experiments to confirm biochemical activity and binding affinity. Additionally, the computational intensity of 70B parameter models remains a hurdle; optimizing smaller, task-specific models is a priority. Finally, while PRO-GO excels at guided generation, designing entirely novel protein folds without existing GO term references remains a challenge. The team plans to integrate molecular dynamics simulations to further validate the stability of generated proteins.
Summary updated