Abstract
Protein sequence design, the inverse problem of protein structure prediction, plays a crucial role in protein engineering. Although recent deep learning-based methods have shown promising advancements, achieving accurate and robust protein sequence design remains an ongoing challenge. Here, we present CarbonDesign, a new approach that draws inspiration from successful ingredients of AlphaFold for protein structure prediction and makes significant and novel developments tailored specifically for protein sequence design. At its core, CarbonDesign explores Inverseformer, a novel network architecture adapted from AlphaFold's Evoformer, to learn representations from backbone structures and an amortized Markov Random Fields model for sequence decoding. Moreover, we incorporate other essential AlphaFold concepts into CarbonDesign: an end-to-end network recycling technique to leverage evolutionary constraints in protein language models and a multi-task learning technique to generate side chain structures corresponding to the designed sequences. Through rigorous evaluations of independent testing data sets, including the CAMEO and recent CASP15 data sets, as well as the predicted structures from AlphaFold, we show that CarbonDesign outperforms other published methods, achieving high accuracy in sequence generation. Moreover, it exhibits superior performance on de novo backbone structures obtained from recent diffusion generative models such as RFdiffusion and FrameDiff, highlighting its potential for enhancing de novo protein design. Notably, CarbonDesign also supports zero-shot prediction of the functional effects of sequence variants, indicating its potential application in directed evolution-based design. In summary, our results illustrate CarbonDesign's accurate and robust performance in protein sequence design, making it a promising tool for applications in bioengineering.
Introduction
Protein sequence design, often referred to as "inverse folding," is the challenge of identifying an amino acid sequence that will fold into a specific, pre-defined 3D backbone structure. This task is fundamental to protein engineering, enabling the creation of novel enzymes, therapeutics, and synthetic materials. While tools like ProteinMPNN and ESM-IF have advanced the field, they often struggle to integrate evolutionary information effectively or handle the structural "noise" inherent in synthetic backbones generated by diffusion models like RFdiffusion. CarbonDesign addresses these gaps by adapting the architectural principles of AlphaFold to the inverse problem.
Methods
CarbonDesign introduces several technical innovations to improve sequence design accuracy and robustness.
The framework consists of several core components:
- Inverseformer: Adapted from AlphaFold’s Evoformer, this module processes 3D structural features—such as local orientations and residue distances—through 12 blocks of node and triangular edge updates. It maps 3D geometry into high-dimensional representations suitable for sequence prediction.
- Amortized Markov Random Fields (MRF): Instead of simple autoregressive decoding, CarbonDesign utilizes an MRF-Sequence Module. This treats sequence design as a probabilistic graphical model problem, capturing complex pairwise dependencies between amino acids.
- End-to-End Recycling with ESM2: The model employs a "recycling" strategy where intermediate outputs are refined through multiple passes. Crucially, it incorporates embeddings from the ESM2 protein language model during these cycles, leveraging evolutionary "common sense" alongside structural constraints.
- Multi-task Learning: The network simultaneously predicts the amino acid sequence and the 3D coordinates of side-chain atoms. This auxiliary task encourages the model to learn physically realistic representations of the protein's internal environment.
Results and Discussion
CarbonDesign was evaluated against standard benchmarks (CAMEO, CASP15), orphan proteins, and de novo structures from diffusion models.
Performance and Robustness
CarbonDesign consistently outperformed state-of-the-art (SOTA) methods. It achieved a 60.1% sequence recovery on the CAMEO dataset, compared to 54.8% for ESM-IF and 51.3% for ProteinMPNN. On de novo backbones from RFdiffusion, CarbonDesign showed a substantial improvement in self-consistency TM-score (scTM), indicating that its designed sequences are more likely to fold into the intended target shapes.
Functional Prediction
In zero-shot prediction of mutation effects, CarbonDesign achieved a Spearman correlation of 0.43, outperforming language models like ESM-1v. It also demonstrated high precision (auROC > 0.95) in distinguishing pathogenic from benign variants in critical genes like TP53.
Theoretical Grounding
A key insight of this work is the connection between the triangular edge updates in the Inverseformer and the Belief Propagation algorithm. This provides a theoretical basis for the architecture's effectiveness in resolving complex spatial constraints. Furthermore, the integration of ESM2 embeddings acts as a powerful prior, ensuring designed sequences remain biologically plausible even when target structures are slightly unnatural.
Conclusion
CarbonDesign represents a significant step forward in robust protein sequence design. By "inverting" the AlphaFold architecture and incorporating large-scale language model priors, it achieves superior accuracy across both natural and synthetic protein backbones. Future work will focus on extending the model to handle multi-chain complexes and validating designed proteins through experimental "wet-lab" synthesis.
Summary updated