TL;DR
This paper introduces a novel method to improve protein stability prediction using existing deep learning models (PROTEINMPNN and ESMIF) without any additional training data or retraining. By leveraging a second term derived from the models themselves, they achieve significant improvements in predicting mutations that increase protein stability, making these powerful tools more accurate for real-world applications.
What Problem Does This Paper Address?
Predicting how a single amino acid change in a protein affects its stability is crucial for protein engineering and understanding disease. Deep learning models like PROTEINMPNN and ESMIF have shown great promise in predicting protein sequences and variant effects. However, their direct application to stability prediction, especially in a "zero-shot" manner (without specific training for this task), can be suboptimal. The challenge lies in extracting the most relevant information from these models for stability prediction without the need for extensive, task-specific experimental data or computationally expensive retraining.
Methods
The core innovation is the introduction of an additional term to the existing prediction framework. Instead of relying solely on the model's output for a mutated protein sequence given its structure, the authors propose to also consider the model's prediction for the single mutated residue in isolation. This "unfolded state" analogy, where the single residue's context is minimal, is then compared to the full context prediction. The difference between these two predictions forms the new term, which is added to the original prediction. This approach effectively leverages the model's internal representations to capture subtle differences related to stability.
They also implemented a "tied decoding" scheme for PROTEINMPNN. This optimization allows the model to reuse computations when predicting each residue in a sequence, significantly reducing the computational overhead of using full sequence context compared to naive methods. This makes proteome-scale predictions feasible.
Datasets
The paper evaluates their method on three established protein stability datasets:
- Tsuboyama et al. dataset: A large saturation mutagenesis dataset correlating with thermal stability.
- S2648 dataset: A curated dataset from the ProTherm database, often used for training ΔΔG predictors.
- S669 dataset: A smaller dataset with low sequence identity to S2648, used as an independent test set.
Key Insights and Ideas
- Leveraging internal model differences is powerful: The most significant insight is that the difference in predictions from a single inverse folding model, when given full context versus minimal (single residue) context, directly correlates with protein stability changes. This suggests that these models implicitly encode stability information that can be unlocked without explicit training.
- Zero-shot improvement is possible: The method demonstrates that substantial performance gains can be achieved without any retraining or additional experimental data, making it a highly practical approach.
- Single residue context as an "unfolded state": The analogy of modeling the single residue as an "unfolded state" in free energy calculations provides a theoretical grounding for their approach.
- Computational efficiency matters: The development of the tied decoding scheme is crucial for enabling large-scale applications, transforming a computationally prohibitive task into a feasible one.
Key Results
- Significant accuracy improvements:
- PROTEINMPNN-DDG achieved an 11% absolute increase in "Success@10" (predicting the top 10 stabilizing mutations) on the Tsuboyama dataset, improving from 66% to 77%.
- ESMIF-DDG showed a 5% absolute increase in "Success@10", improving from 64% to 69%.
- Similar improvements were observed across other metrics (auROC, auPRC) and datasets.
- Computational speedup: The tied decoding scheme for PROTEINMPNN reduces the slowdown from over 200-fold to under 4-fold for long sequences, enabling proteome-scale predictions. This results in a throughput of 9800 residues/second on a V100 GPU, a massive improvement over existing methods.
Limitations and Future Work
- Methionine bias in ESMIF: The paper notes a bias in ESMIF's single-residue predictions towards methionine due to its prevalence in the training data. They developed a correction for this bias, demonstrating its importance.
- Generalizability beyond stability: The authors suggest that the core approach of leveraging differences in model outputs from varying input contexts could be extended to other protein property prediction tasks, such as predicting binding affinity changes.
- Further optimization: While significant, the tied decoding scheme still introduces a slowdown. Further optimizations might be explored.
Summary updated