LaGDif: Latent Graph Diffusion Model for Efficient Protein Inverse Folding with Self-Ensemble

Tai Te Wu, Yu Guang Wang, Yiqing ShenBIBM 2024

Abstract

Protein inverse folding aims to identify viable amino acid sequences that can fold into given protein structures, enabling the design of novel proteins with desired functions for applications in drug discovery, enzyme engineering, and biomaterial development. Diffusion probabilistic models have emerged as a promising approach in inverse folding, offering both feasible and diverse solutions compared to traditional energy-based methods and more recent protein language models. However, existing diffusion models for protein inverse folding operate in discrete data spaces, necessitating prior distributions for transition matrices and limiting smooth transitions and gradients inherent to continuous spaces, leading to suboptimal performance. Drawing inspiration from the success of diffusion models in continuous domains, we introduce the Latent Graph Diffusion Model for Protein Inverse Folding (LaGDif). LaGDif bridges discrete and continuous realms through an encoder-decoder architecture, transforming protein graph data distributions into random noise within a continuous latent space. Our model then reconstructs protein sequences by considering spatial configurations, biochemical attributes, and environmental factors of each node. Additionally, we propose a novel inverse folding self-ensemble method that stabilizes prediction results and further enhances performance by aggregating multiple denoised output protein sequence. Empirical results on the CATH dataset demonstrate that LaGDif outperforms existing state-of-the-art techniques, achieving up to 45.55% improvement in sequence recovery rate for single-chain proteins and maintaining an average RMSD of 1.96 Å between generated and native structures.

Introduction

In protein engineering, the "inverse folding" problem is a fundamental challenge: given a specific 3D structure that performs a desired function, what amino acid sequence will fold into that shape? Traditional approaches often struggle with the "one-to-many" nature of this problem, where multiple sequences can fold into the same structure. While recent diffusion models have shown promise, they typically operate in discrete spaces, treating amino acids as distinct tokens. This discrete approach creates "jumpy" optimization landscapes that limit performance.

LaGDif (Latent Graph Diffusion) addresses these hurdles by moving the entire diffusion process into a continuous latent space. By mapping sequences into a high-dimensional continuous domain, the model allows for smoother transitions and more effective exploration of the sequence space, significantly advancing the state-of-the-art in structural fidelity and sequence diversity.

Methodology

The LaGDif architecture is built upon three core technical pillars designed to bridge the gap between discrete protein data and continuous diffusion math.

Latent Graph Diffusion

Instead of diffusing raw amino acid labels, LaGDif utilizes a pre-trained ESM2 encoder to map sequences into a continuous latent space. Diffusion—the process of adding and removing noise—occurs within this high-dimensional space. This allows the model to inherit a "biological prior," leveraging the semantic relationships between amino acids already captured by large-scale protein language models. A linear decoder eventually maps the denoised latent vectors back to discrete amino acids.

The LaGDif architecture overview, illustrating the transition from discrete protein graphs to a continuous latent space for diffusion and back.

Equivariant Denoising and Guided Sampling

The denoiser at the heart of LaGDif is an Equivariant Graph Neural Network (EGNN). Because proteins are 3D objects, the model must be rotationally and translationally invariant; the EGNN ensures that the predicted sequence remains consistent regardless of the protein's orientation in space. To maintain structural integrity, the model employs guided sampling, starting from a point closer to the target distribution rather than pure static noise.

Self-Ensemble Strategy

A standout feature of LaGDif is its self-ensemble method. During the iterative denoising process, the model generates $K$ candidate sequences at each step and averages their features in the latent space. This "wisdom of the crowd" approach acts as a regularizer, stabilizing predictions and filtering out individual errors, which leads to significantly more "foldable" sequences.

Results and Discussion

LaGDif was evaluated on the CATH dataset (version 4.2.0), a standard benchmark split into short proteins, single-chain proteins, and complex multi-chain structures.

Performance Metrics

LaGDif sets a new benchmark across several critical metrics:

  • Sequence Recovery: The model achieved a recovery rate of approximately 88% across categories. This represents a massive leap from the previous state-of-the-art (Grade-IF), which sat around 52%.
  • Structural Fidelity: When predicted sequences were folded using ESMFold, they matched target structures with an average Root Mean Square Deviation (RMSD) of 1.96 Å. An RMSD under 2 Å is considered a high-quality match in structural biology.
  • Confidence: The model demonstrated lower perplexity (~2.5) compared to competitors (~4.5), indicating higher certainty in its generated sequences.
Visual comparison of LaGDif against state-of-the-art models on the CATH dataset, highlighting superior sequence recovery and structural alignment.

Limitations and Future Directions

While highly accurate, LaGDif's iterative diffusion process makes it slower than "one-shot" models like ProteinMPNN (2.95s vs 0.45s per protein). It also has a larger memory footprint (~280 MB). Future work will focus on extending this latent graph diffusion approach to de novo protein design and complex protein-protein interaction predictions.

Reference: Wu, T., Wang, Y. G., & Shen, Y. (2024). LaGDif: Latent Graph Diffusion Model for Efficient Protein Inverse Folding with Self-Ensemble. arXiv:2411.01737v1.

Summary updated

Abstract

Protein inverse folding aims to identify viable amino acid sequences that can fold into given protein structures, enabling the design of novel proteins with desired functions for applications in drug discovery, enzyme engineering, and biomaterial development. Diffusion probabilistic models have emerged as a promising approach in inverse folding, offering both feasible and diverse solutions compared to traditional energy-based methods and more recent protein language models. However, existing diffusion models for protein inverse folding operate in discrete data spaces, necessitating prior distributions for transition matrices and limiting smooth transitions and gradients inherent to continuous spaces, leading to suboptimal performance. Drawing inspiration from the success of diffusion models in continuous domains, we introduce the Latent Graph Diffusion Model for Protein Inverse Folding (LaGDif). LaGDif bridges discrete and continuous realms through an encoder-decoder architecture, transforming protein graph data distributions into random noise within a continuous latent space. Our model then reconstructs protein sequences by considering spatial configurations, biochemical attributes, and environmental factors of each node. Additionally, we propose a novel inverse folding self-ensemble method that stabilizes prediction results and further enhances performance by aggregating multiple denoised output protein sequence. Empirical results on the CATH dataset demonstrate that LaGDif outperforms existing state-of-the-art techniques, achieving up to 45.55% improvement in sequence recovery rate for single-chain proteins and maintaining an average RMSD of 1.96 Å between generated and native structures. The code is public available at https://github.com/TaoyuW/LaGDif.
BibTeX
@inproceedings{wu2024lagdif,
  author = {Tai Te Wu and Yu Guang Wang and Yiqing Shen},
  title = {{LaGDif: Latent Graph Diffusion Model for Efficient Protein Inverse Folding with Self-Ensemble}},
  booktitle = {BIBM},
  year = {2024},
}
Figure