Improving protein tertiary structure prediction by deep learning and distance prediction in <scp>CASP14</scp>

Jian Liu, Tianqi Wu, Zhiye Guo, Jie Hou, Jianlin ChengProteins 2021 · DOI

TL;DR

This paper introduces the MULTICOM system for protein tertiary structure prediction, significantly improving accuracy by integrating deep learning-based inter-residue distance prediction. The enhanced system performs competitively in CASP14, demonstrating that template-free modeling can outperform template-based methods even for targets with known homologous structures.

What Problem Does This Paper Address?

Predicting the 3D structure of a protein from its amino acid sequence is a fundamental challenge in biology with vast implications for understanding protein function, drug design, and disease mechanisms. Traditional methods fall into two categories: template-based modeling (TBM), which uses known protein structures as templates, and template-free (ab initio) modeling, which attempts to build structures from scratch. While TBM is effective when suitable templates exist, it fails for novel protein folds. Ab initio methods have historically struggled with accuracy, especially for larger proteins. The Critical Assessment of protein Structure Prediction (CASP) experiments serve as a benchmark for evaluating these methods. This paper addresses the challenge of improving protein structure prediction accuracy, particularly for difficult "hard" targets (those lacking clear templates) and aims to push the boundaries of both template-free and template-based approaches.

Methods

The MULTICOM system for CASP14 was enhanced with three key components:

  • Deep Learning-Based Inter-Residue Distance Predictor (DeepDist): This new component uses deep learning to predict the distances between pairs of amino acid residues. This is crucial for guiding the folding process in template-free modeling.
  • Enhanced Template-Based Tertiary Structure Prediction: The TBM pipeline was streamlined by removing redundant tools and incorporating deeper multiple sequence alignments (MSAs) for more robust template searching.
  • Deep Learning-Powered Model Quality Assessment (QA): New QA methods, including DeepRank variants (DeepRank3_Cluster and DeepRank_con), were developed to better assess and rank predicted protein models. These methods leverage predicted distance maps from DeepDist.

The overall pipeline involves domain parsing, parallel template-based and template-free modeling, model preprocessing, ranking, and final model generation.

Datasets

The primary dataset used for evaluation is the set of targets from the CASP14 experiment. The paper refers to official evaluation data downloaded from the CASP14 website. For training the DeepDist predictor, the paper mentions using MSAs generated by tools like DeepMSA and DeepAln, and potentially incorporating data from the Big Fantastic Database (BFD) which includes sequences from metagenome databases. The quality assessment methods were trained and tested on models from previous CASP experiments.

Key Insights and Ideas

  • Template-Free Modeling's Rise: The study strongly suggests that template-free modeling, particularly when powered by accurate deep learning-based distance predictions, can achieve accuracy comparable to or even exceeding template-based methods, even for targets with homologous templates. This blurs the traditional distinction between these two approaches.
  • The Critical Role of MSAs: The accuracy of distance prediction, and consequently the quality of the final predicted structure, is heavily reliant on the quality of the multiple sequence alignments (MSAs). Higher quality MSAs (indicated by a higher number of effective sequences, Neff) lead to more accurate distance predictions and better structural models.
  • Domain Prediction is Key for Multidomain Proteins: For proteins composed of multiple domains, accurate identification of domain boundaries is crucial. Errors in domain prediction can significantly impact the accuracy of the final tertiary structure prediction.
  • QA Methods Need Improvement for Hard Targets: While the new deep learning-based QA methods show promise, they still struggle with "hard" targets where the distribution of model quality scores is highly skewed, and few good models are available. This indicates a need for more robust QA strategies for challenging cases.

Key Results

  • CASP14 Performance: The MULTICOM predictor was ranked seventh out of 146 predictors in tertiary structure prediction and third out of 136 predictors in inter-domain structure prediction at CASP14.
  • Template-Free vs. Template-Based: For FM/TBM domains, MULTICOM-DIST (template-free) achieved an average TM-score of 0.513, significantly outperforming MULTICOM-HYBRID (template-based) with an average TM-score of 0.211. For TBM domains, MULTICOM-DIST (template-free) achieved an average TM-score of 0.703, also outperforming MULTICOM-HYBRID (template-based) with an average TM-score of 0.636.
  • Accuracy on TBM Targets: MULTICOM-DIST predicted the correct fold for 91.4% of TBM domains when considering the top-1 model, increasing to 96.6% when considering the best of five models.
  • MSA Quality Impact: A strong positive correlation was observed between the number of effective sequences (Neff) in MSAs and model quality, particularly for Neff values up to around 400. Similarly, the precision of distance prediction strongly correlated with model quality.
  • Server Performance: MULTICOM-DEEP and MULTICOM-HYBRID showed competitive performance, with MULTICOM-DEEP achieving a correct fold prediction for 78.1% of domains.

Limitations and Future Work

  • QA for Hard Targets: The paper acknowledges that current QA methods, even the new deep learning-based ones, perform poorly when only a few good models are predicted for a hard target and the distribution of model quality scores is highly skewed. This suggests a need for improved QA methods that can better handle such challenging scenarios.
  • Large Multidomain Proteins: For very large multidomain proteins, accurately predicting residue-residue distances across the entire sequence can be challenging due to difficulties in obtaining comprehensive MSAs. This can lead to poor predictions, as seen with target T1036s1. The authors suggest that incorporating template-based distance information might be beneficial for such cases.
  • Domain Prediction Accuracy: The accuracy of domain prediction significantly impacts the overall tertiary structure prediction for multidomain proteins. Future work could focus on further improving domain prediction algorithms.
  • Future Directions: The authors suggest that the traditional division of protein structure prediction into template-based and template-free modeling may become less relevant as template-free methods become more powerful. They also highlight the need for better methods to utilize distance predictions within quality assessment.

Summary updated

Abstract

Substantial progresses in protein structure prediction have been made by utilizing deep-learning and residue-residue distance prediction since CASP13. Inspired by the advances, we improve our CASP14 MULTICOM protein structure prediction system by incorporating three new components: (a) a new deep learning-based protein inter-residue distance predictor to improve template-free (ab initio) tertiary structure prediction, (b) an enhanced template-based tertiary structure prediction method, and (c) distance-based model quality assessment methods empowered by deep learning. In the 2020 CASP14 experiment, MULTICOM predictor was ranked seventh out of 146 predictors in tertiary structure prediction and ranked third out of 136 predictors in inter-domain structure prediction. The results demonstrate that the template-free modeling based on deep learning and residue-residue distance prediction can predict the correct topology for almost all template-based modeling targets and a majority of hard targets (template-free targets or targets whose templates cannot be recognized), which is a significant improvement over the CASP13 MULTICOM predictor. Moreover, the template-free modeling performs better than the template-based modeling on not only hard targets but also the targets that have homologous templates. The performance of the template-free modeling largely depends on the accuracy of distance prediction closely related to the quality of multiple sequence alignments. The structural model quality assessment works well on targets for which enough good models can be predicted, but it may perform poorly when only a few good models are predicted for a hard target and the distribution of model quality scores is highly skewed. MULTICOM is available at https://github.com/jianlin-cheng/MULTICOM_Human_CASP14/tree/CASP14_DeepRank3 and https://github.com/multicom-toolbox/multicom/tree/multicom_v2.0.
BibTeX
@inproceedings{liu2021improving,
  author = {Jian Liu and Tianqi Wu and Zhiye Guo and Jie Hou and Jianlin Cheng},
  title = {{Improving protein tertiary structure prediction by deep learning and distance prediction in <scp>CASP14</scp>}},
  booktitle = {Proteins},
  year = {2021},
  doi = {10.1002/prot.26186},
}
Figure