Miniaturizing, Modifying, and Magnifying Nature’s Proteins with Raygun

Kapil Devkota, Daichi Shonai, J. G. Mao, Scott H. Soderling, Rohit SinghbioRxiv 2024 · DOI

TL;DR

Raygun is a new AI framework that can efficiently engineer proteins by miniaturizing, modifying, or expanding them while preserving their function and structure. It achieves this by representing proteins as probability distributions, enabling unprecedented control over protein design and opening doors for new biotechnological tools and therapeutics.

What Problem Does This Paper Address?

Nature has evolved proteins over billions of years through a complex interplay of substitutions, insertions, and deletions (indels). While computational protein design has made strides, it still struggles to replicate this natural evolutionary capability, especially when it comes to making large-scale changes like adding or removing entire protein segments (indels) while maintaining function. Existing methods often rely on point substitutions, which are limited in scope and can lead to an explosion of possibilities that are computationally intractable. The challenge is to develop a design approach that can handle both extensive mutations and indels, allowing for the creation of novel protein variants from existing templates.

Methods

Raygun employs a novel approach to protein design by leveraging protein language models (PLMs). Instead of representing a protein as a single point in a high-dimensional space, Raygun encodes it as a probability distribution, specifically a multivariate normal (MVN) distribution. This allows for a fixed-size representation that is independent of protein length.

The core of Raygun is an autoencoder architecture:

  • Encoder: This component takes a variable-length protein sequence and maps it to a fixed-size MVN distribution. It achieves this by dividing the protein's PLM embeddings into segments and modeling each segment's distribution.
  • Decoder: This component takes the fixed-size MVN distribution and reconstructs a variable-length protein sequence.

Raygun's design parameters, noise and desired length, offer fine-grained control. The noise parameter influences the rate of substitutions, while the desired length dictates the extent of indels. This "single-shot" generation approach is significantly faster than diffusion-based methods. The authors also incorporate a recycling mechanism, where generated candidates are fed back into the model to improve diversity and quality.

Datasets

  • UniRef-50: Used for pre-training Raygun, comprising approximately 94,734 proteins with diverse lengths.
  • PFAM database: Used for evaluating Raygun's ability to preserve functional domains across various structural classes.
  • FPbase: Used to compare the size of generated fluorescent proteins against a database of known FPs.
  • GFP brightness dataset: Used to train a model for selecting brighter fluorescent protein candidates.
  • CATH database: Used to evaluate the clustering performance of Raygun's fixed-length representations.

Key Insights and Ideas

  • Proteins as Probability Distributions: Representing proteins as probability distributions, rather than fixed points, is a fundamental shift that allows for length-independent encoding and more flexible generation.
  • Bridging Evolution and AI: Raygun's approach connects evolutionary principles (like indels) with AI capabilities (PLMs), creating a powerful framework for protein engineering.
  • Template-Guided Design is Powerful: The paper demonstrates that starting from existing proteins (template-guided design) can be highly effective for generating novel variants with desired properties, even for functions not explicitly trained for.
  • Function is More Than Sequence Similarity: The EGF binder results suggest that Raygun can identify functional improvements even when sequence identity to the wildtype is not maximized, hinting at deeper functional attributes being captured.

Key Results

  • Miniaturization: Raygun can shrink proteins by 10-25%, and in some cases over 50%, while preserving structural integrity (e.g., fluorescent proteins smaller than 96% of those in FPbase).
  • Modification & Augmentation: Raygun can introduce extensive sequence diversity and expand proteins beyond their natural size.
  • Functional Preservation: Raygun preferentially preserves active and binding sites during miniaturization, with a slight preference for deleting less-structured loop regions.
  • Speed: Raygun's single-shot generation is ~100x faster than diffusion-based de novo design approaches.
  • Experimental Validation:
    • Miniaturized fluorescent proteins showed fluorescence.
  • TurboID variants were expressed, with two showing slight biotin ligase activity.
  • EGF variants showed stronger binding affinity to EGFR than the wildtype.
  • Structural Versatility: Raygun effectively preserves protein structure across a wide range of lengths and structural classes.
  • Representation Quality: Raygun's fixed-length representations better capture the hierarchical organization of protein structure space compared to standard PLM embeddings.

Limitations and Future Work

  • TurboID Miniaturization: While Raygun could remove the DNA-binding domain for extreme miniaturization of TurboID, the ligase activity was not significant. The authors suggest that directed evolution or hybrid approaches might be needed to optimize function in such cases.
  • Non-Canonical Chromophores: For fluorescent proteins, Raygun generated candidates with non-canonical chromophores, which were dimmer. Further directed evolution might be needed to enhance their brightness.
  • Fine-tuning for Specific Tasks: While Raygun is general-purpose, fine-tuning on specific protein families or tasks can improve performance.
  • Future Work: The authors suggest incorporating more powerful PLMs, larger training sets, or structure-infused models to further enhance Raygun's capabilities. They also highlight the potential for Raygun in drug discovery, synthetic biology, and other areas of computational biology.

Summary updated

Abstract

Abstract Proteins have evolved over billions of years through extensive and coordinated substitutions, insertions and deletions (indels). Computational protein design cannot yet fully mimic nature’s ability to engineer new proteins from existing templates. Protein language models generate informative per-residue representations, but leveraging them to execute large-scale, function-preserving mutations and indels has remained beyond reach. We introduce Raygun, a generative AI framework that unlocks efficient miniaturization, modification, and augmentation of proteins, using a novel probabilistic encoding of protein sequences constructed from language model embeddings. Emulating evolution, Raygun shrinks proteins by 10-25% (sometimes over 50%) while preserving predicted structural integrity and fidelity, introduces extensive sequence diversity while preserving functional sites, and can expand proteins beyond their natural size. These capabilities unlock new opportunities in gene therapy and biotechnology. In cell-based validation, Raygun successfully miniaturized fluorescent proteins, two of which are smaller than 96% of fluorescent proteins reported in FPbase, as well as TurboID, a synthetic biotin ligase widely adopted for proteomics. It also successfully expanded Epidermal Growth Factor (EGF), a natural binding partner to the EGFR protein, generating EGF variants with higher binding affinity than the wildtype. Raygun’s conceptual innovations in template-based protein design reveal that protein function can be encoded in a length-independent space. This fundamental insight bridges protein representation learning with evolutionary biology and could unlock the development of more efficient molecular tools and biological therapeutics.
BibTeX
@misc{devkota2024miniaturizing,
  author = {Kapil Devkota and Daichi Shonai and J. G. Mao and Scott H. Soderling and Rohit Singh and Scott Soderling and Rohit Singh},
  title = {{Miniaturizing, Modifying, and Magnifying Nature’s Proteins with Raygun}},
  year = {2024},
  doi = {10.1101/2024.08.13.607858},
  howpublished = {bioRxiv},
}
Figure