SE(3) Equivalent Graph Attention Network as an Energy-Based Model for Protein Side Chain Conformation

Liu De-qin, Sheng Chen, Shuangjia Zheng, Sen Zhang, Yuedong Yang

TL;DR

This paper introduces GraphEBM, a novel SE(3)-equivariant graph attention network designed to learn protein energy functions directly from data. By outperforming existing methods on rotamer recovery and showing promise in combinatorial side chain optimization, GraphEBM offers a more data-driven and potentially less biased approach to understanding protein conformations.

What Problem Does This Paper Address?

Designing and predicting protein structures is a fundamental challenge in biology and medicine. A key aspect of this is understanding how amino acid side chains, which are crucial for protein function and interactions, arrange themselves in 3D space. Traditionally, this has relied on energy functions that approximate physical forces and incorporate expert-derived features. However, this manual feature engineering can be biased and may miss complex, high-order interactions. The authors aim to overcome this by developing a deep learning model that can learn these energy functions directly from experimental data (crystal structures), thereby discovering unknown features and reducing reliance on human expertise. Specifically, they focus on the problem of predicting the most stable conformation of a protein's side chain given its local backbone structure.

Methods

The core of GraphEBM is a graph neural network architecture that leverages SE(3) equivariance. SE(3) is the group of rotations and translations in 3D space, and equivariance means the model's output (the predicted energy) transforms predictably when the input (the protein conformation) is rotated or translated. This is a crucial property for molecular modeling, as the absolute position and orientation of a molecule shouldn't affect its intrinsic energy.

GraphEBM builds upon the DimeNet++ architecture, which uses radial basis functions (RBFs) and spherical Bessel functions (SBFs) to encode geometric relationships between atoms. The authors enhance this by:

  • Integrating Graph Attention Networks (GAT): GATs are used to capture topological information and relationships between atom types and residue types, which were not fully utilized in DimeNet++.
  • Introducing a Smooth Factor: To address gradient explosion issues during training, especially when atoms are very close, they add a smoothing factor to the RBF and SBF calculations. This stabilizes the training process.
  • Energy-Based Model (EBM) Framework: GraphEBM is framed as an energy-based model, meaning it learns a function that assigns a scalar energy value to each possible conformation. The goal is to assign lower energies to more stable (native-like) conformations.

Datasets

The study uses a dataset derived from high-resolution PDB structures, similar to that used in prior work [23]. The training dataset contains 12,473 structures, and the test dataset comprises 121 structures.

Key Insights and Ideas

  • Data-Driven Energy Functions are Viable: The success of GraphEBM demonstrates that deep learning models can effectively learn complex protein energy functions directly from structural data, potentially surpassing hand-crafted features.
  • SE(3) Equivariance is Crucial: The paper highlights the importance of SE(3) equivariance for molecular modeling, ensuring that the model's predictions are physically consistent regardless of the molecule's orientation.
  • GAT Enhances Geometric Reasoning: Incorporating GATs allows the model to better leverage the rich information present in atom and residue types, complementing the geometric encoding of RBFs and SBFs.
  • Smoothness for Stability: The introduction of a smooth factor is a practical and effective technique for stabilizing the training of models dealing with potentially problematic geometric inputs (e.g., very close atoms).
  • Surface Residues are More Challenging: The results show that GraphEBM, like other methods, performs better on buried residues than surface residues. This is attributed to the reduced physical constraints and increased flexibility of surface residues.

Key Results

  • Rotamer Recovery: GraphEBM outperformed both Rosetta's energy functions and the state-of-the-art Atom Transformer on the side chain rotamer recovery task, achieving higher accuracy across different sampling strategies (discrete and continuous). For example, on the continuous sampling strategy, GraphEBM achieved an average recovery rate of 78.4%, compared to 74.1% for the Atom Transformer and 75.8% for Rosetta ref2015.
  • Combinatorial Side Chain Optimization: When applied to optimizing side chains for a fixed backbone (using an initialization from PULCHRA), GraphEBM improved the χ1 rotamer recovery by 22.2% on average compared to the PULCHRA method alone.
  • Ablation Studies: Removing the smooth factor or the GAT component led to a decrease in rotamer recovery performance, underscoring their importance.

Limitations and Future Work

  • Sampling Strategy Dependence: The authors note that the performance is influenced by the sampling strategy used for generating candidate conformations.
  • Combinatorial Optimization Challenges: While GraphEBM shows promise in combinatorial side chain optimization, the current iterative approach is described as "naive." The paper suggests that generating side chain conformations directly might be a more fruitful future direction.
  • Generalizability: The current work focuses on side chain conformation. Extending this to full protein structure prediction or design remains an open area.
  • DFT Connection: The authors hint at a deeper connection to Density Functional Theory (DFT) and suggest its potential for further applications in deep learning for molecular systems.

Summary updated

Abstract

Abstract Protein design energy functions have been developed over decades by leveraging physical forces approximation and knowledge-derived features. However, manual feature engineering and parameter tuning might suffer from knowledge bias. Learning potential energy functions fully from crystal structure data is promising to automatically discover unknown or high-order features that contribute to the protein’s energy. Here we proposed a graph attention network as an energy-based model for protein conformation, namely GraphEBM. GraphEBM is equivariant to the SE(3) group transformation, which is the important principle of modern machine learning for molecules-related tasks. GraphEBM was benchmarked on the rotamer recovery task and outperformed both Rosetta and the state-of-the-art deep learning based methods. Furthermore, GraphEBM also yielded promising results on combinatorial side chain optimization, improving 22.2% χ 1 rotamer recovery to the PULCHRA method on average.
BibTeX
@inproceedings{deqin2023se,
  author = {Liu De-qin and Sheng Chen and Shuangjia Zheng and Sen Zhang and Yuedong Yang},
  title = {{SE(3) Equivalent Graph Attention Network as an Energy-Based Model for Protein Side Chain Conformation}},
  booktitle = {BIBM},
  year = {2023},
  url = {https://openreview.net/forum?id=NRsCR34Jl1},
}
Figure