Abstract
Understanding protein sequence-to-function relationship is crucial to assist studies of genetic diseases, protein evolution, and protein engineering. The sequence-to-function relationship of proteins is inherently complex due to multi-site high-dimensional correlation and structural dynamics. Deep learning algorithms such as (graph) convolutional neural networks and recently transformers have become very popular for learning the protein sequence-to-function mapping from deep mutational scanning data and available structures. However, it remains very challenging for these models to achieve accurate extrapolation when predicting functional effect of variants with positions or mutation types not seen in the training data. We propose that incorporating the physics of protein interactions and dynamics can be an effective approach to overcome the extrapolation limitations. Specifically, we demonstrate that physics-based modeling can be used to quantify the energetic effects of mutations and that incorporating these physical energetics directly within the convolution and graph convolution neural networks can significantly improve the performance of positional and mutational extrapolation compared to models without biophysics-inspired features. Our results support the effectiveness of leveraging physical knowledge in overcoming the limitation of data scarcity.
Introduction
In protein engineering, predicting how specific mutations alter a protein's function is a fundamental challenge. Deep Mutational Scanning (DMS) has enabled the generation of large-scale datasets, but the astronomical size of sequence space means even the most comprehensive experiments cover only a fraction of possible variants. While standard deep learning models excel at interpolation—predicting effects similar to those in the training set—they struggle with extrapolation. Specifically, they often fail when predicting the effects of mutation types not seen during training (mutational extrapolation) or mutations at entirely new positions (positional extrapolation). This "extrapolation gap" arises because these models lack an inherent understanding of the physical laws governing protein folding and stability.
Methods: Integrating Physics into Neural Networks
To address these limitations, we developed a hybrid approach that augments standard neural network architectures with biophysical features.
Physics-Based Features
- Rosetta Energetics: We utilized the Rosetta software suite to calculate the change in folding stability ($\Delta\Delta G$) for all possible single-point mutations. Rather than using a single aggregate score, we decomposed the energy into eight distinct components, including hydrogen bonding, electrostatic potential, and steric clashes.
- Molecular Dynamics (MD): To account for protein flexibility, we performed 200-nanosecond simulations to calculate the Root Mean Square Fluctuation (RMSF), identifying highly dynamic regions of the protein structure.
Model Architectures
These features were integrated into two primary architectures: Convolutional Neural Networks (CNNs), which treat the protein sequence as a 1D input, and Graph Convolutional Networks (GCNs), which represent the protein as a 3D graph where residues are nodes connected based on spatial proximity in the folded structure.
Datasets and Benchmark Proteins
The proposed models were evaluated across five diverse proteins, representing a range of sizes, functions, and data coverage levels.
The proteins range from the small, well-covered GB1 domain to the larger, more complex Bgl3 enzyme. This diversity allows for a robust assessment of how physics-informed features handle varying degrees of data scarcity and structural complexity.
Key Results and Insights
The incorporation of physical priors significantly enhanced the models' ability to generalize beyond the training data.
Overcoming the Extrapolation Barrier
- Positional Extrapolation: In tasks where models had to predict mutations at completely unseen sites, standard architectures often showed near-zero correlation. Adding physics features improved the CNN's Pearson correlation on GB1 from 0.05 to 0.43.
- Mutational Extrapolation: For avGFP, the addition of physics features boosted correlation from 0.06 to 0.61, demonstrating that physical rules (like steric hindrance) provide a crucial "grounding" for the model.
Efficiency and Data Scarcity
Physics-informed models proved particularly resilient in data-scarce scenarios. By providing the model with "cheat sheets" of energetic costs, it can learn meaningful patterns from fewer experimental examples. Furthermore, this approach achieved performance comparable to massive Transformer-based models (e.g., METL) pre-trained on millions of sequences, suggesting that explicit physical knowledge can be as effective as massive-scale data mining.
Conclusion and Future Work
While incorporating physics significantly improves extrapolation, challenges remain. For instance, the dynamic nature of proteins like Ube4b still poses difficulties for current physics-based tools. Future research will focus on moving beyond the assumption of mutational additivity to better capture epistasis—the complex interactions between multiple mutations. Nonetheless, this work demonstrates that bridging the gap between deep learning and biophysics is a powerful strategy for navigating the vast landscape of protein sequence space.
Summary updated