An all-atom protein generative model

Alexander E. Chu, Jinho Kim, Lucy Cheng, Gina El Nesr, Minkai Xu, Richard W. Shuai, Po-Ssu HuangProceedings of the National Academy of Sciences 2024 · OpenReview

TL;DR

Researchers have developed Protpardelle, a novel all-atom generative model for proteins that can design both the protein's structure and its amino acid sequence simultaneously. This breakthrough allows for more precise control over protein design by considering all atoms, including sidechains, which are crucial for function, paving the way for designing proteins with entirely new functions.

What Problem Does This Paper Address?

Designing proteins with novel functions is a grand challenge in biology, medicine, and engineering. While protein sequence dictates its structure and function, accurately modeling the intricate chemical interactions, particularly those mediated by sidechains, has been a significant hurdle. Existing protein design methods often simplify this by focusing on the protein backbone and then inferring sidechain arrangements, or by designing sequence and structure separately. This approach limits the ability to precisely control and design for specific functional interactions, as sidechains play a critical role in everything from enzyme catalysis to protein-protein binding. The paper addresses this by proposing an "all-atom" generative model that can design the complete protein, including every atom in the backbone and all sidechains, in a unified process.

Methods

The core of Protpardelle is a diffusion model, adapted from image generation, that operates on a novel "superposition" state. This superposition represents all possible sidechain conformations for each amino acid position simultaneously. During the generative process, this superposition is iteratively "collapsed" into a specific amino acid and conformation, guided by a learned score function. This approach allows the model to handle the inherent uncertainty in sequence and sidechain identity during generation.

Key methodological innovations include:

  • All-Atom Superposition State: Instead of modeling discrete amino acids and their specific atoms, the model represents a "superposition" of all 20 possible sidechains at each position, along with the backbone atoms. This allows for a continuous denoising process that can accommodate evolving sequence predictions.
  • Joint Structure and Sequence Codesign: The model integrates a sequence prediction module (a modified ProteinMPNN) that estimates the amino acid sequence at each denoising step. This sequence prediction then informs the collapse of the sidechain superposition, enabling codesign of structure and sequence.
  • Probability Flow ODE Formulation: The diffusion process is framed as solving a probability flow ordinary differential equation (ODE), which offers advantages in terms of deterministic sampling and connection to continuous normalizing flows.
  • Two-Stage Sampling: To improve the quality of generated sidechains, a two-stage sampling process is employed. The first stage generates the backbone and sequence, and a second stage refines the sidechains, effectively acting as a rotamer packing step.

Datasets

The paper primarily uses protein structures from the Protein Data Bank (PDB) for training and evaluation. For some experiments, they also incorporate data from the AlphaFold Protein Structure Database (AFDB). No new benchmark datasets were introduced.

Key Insights and Ideas

  • Sidechains are Key: The paper strongly emphasizes that sidechains are the primary drivers of protein function and that modeling them explicitly is crucial for advanced protein design.
  • Superposition as a Solution to Uncertainty: The novel "superposition" state elegantly handles the uncertainty inherent in predicting both sequence and structure simultaneously, a major challenge in all-atom protein generation.
  • Diffusion Models Adapt Well to Proteins: The success of adapting diffusion models, originally developed for images, to the complex, discrete-continuous nature of protein structures highlights the versatility of these generative frameworks.
  • Computational Efficiency Matters: The authors highlight the computational efficiency of their model, particularly the backbone-only version, which is orders of magnitude faster than previous methods, enabling more rapid exploration of design space.
  • Codesign is More Than Just Sequence-Structure Alignment: The ability to influence the backbone based on sidechain predictions and vice-versa (as demonstrated in repacking tasks) suggests a deeper level of integration than previously achieved.

Key Results

  • High-Quality Samples: Protpardelle generates proteins that meet typical quality, diversity, and novelty metrics.
  • Improved Designability: The generated proteins exhibit good self-consistency, meaning their designed sequences are predicted to fold into structures similar to the generated ones. For backbone-only generation, success rates (scRMSD < 2) are ≥90% for most lengths.
  • Computational Speed: The backbone-only model generates structures at a rate of 0.1-0.5 s per 100 residues, significantly faster than previous methods (15-60 s per 100 residues). The all-atom model is approximately ten-fold slower but still efficient.
  • Novelty and Diversity: Samples show good diversity in secondary structure content and demonstrate the ability to generate novel folds not present in the training set.
  • Plausible Sidechain Chemistry: Analysis of bond lengths, angles, and chi angles shows that generated sidechains closely resemble those of natural proteins, with reasonable fa_dun energies.
  • Functional Motif Scaffolding: The model shows promise in scaffolding functional motifs, even when conditioned only on specific chemical groups, suggesting potential for designing proteins with tailored interactions.

Limitations and Future Work

  • Sequence Leakage: The model can suffer from "sequence leakage," where the model memorizes the structure given a sequence, leading to a distribution shift. This is partially mitigated by noising all atom positions.
  • All-Atom Model Robustness: The all-atom model is less robust than the backbone-only model, particularly at longer lengths, suggesting that modeling sidechains adds significant complexity.
  • Preference for Certain Structures: The all-atom model shows a relative preference for proteins with ~50% helix and ~20% sheet content, which may be due to dataset biases or sequence leakage.
  • ESMFold Self-Consistency for Conditional Designs: While some conditional designs are plausible by inspection, they do not yet pass ESMFold-based self-consistency thresholds, indicating room for improvement in fidelity.
  • Future Work: The authors suggest exploring the model's potential for more advanced applications like protein-protein interaction design, exploring the relationship between model likelihoods and energy landscapes, and developing more robust sequence codesign strategies. They also note that while their model is structure-primary, future work could explore joint sequence-structure generation and inverse folding.

Summary updated

BibTeX
@inproceedings{chu2024all,
  author = {Alexander E. Chu and Jinho Kim and Lucy Cheng and Gina El Nesr and Minkai Xu and Richard W. Shuai and Po-Ssu Huang},
  title = {{An all-atom protein generative model}},
  booktitle = {Proceedings of the National Academy of Sciences},
  year = {2024},
  url = {https://openreview.net/forum?id=sHCmPyD60o},
}
Figure