Fold recognition by scoring protein map similarities using the congruence coefficient

Pietro Di Lena, Pierre BaldibioRxiv 2020 · DOI

TL;DR

This paper introduces the congruence coefficient as a novel and mathematically robust metric for comparing protein contact and distance maps. By leveraging its statistical properties, the authors demonstrate that this coefficient significantly improves protein fold recognition accuracy compared to existing methods, offering a more effective way to identify distantly related protein structures.

What Problem Does This Paper Address?

Predicting the 3D structure of proteins is a fundamental challenge in biology. A key step in many prediction methods is "fold recognition," which aims to identify known protein structures (templates) that are similar to a target protein, even if they are not obviously related through sequence homology. While sequence-based methods work well for closely related proteins, identifying distantly related ones is much harder. Recent advances in predicting residue-residue contacts and distances from protein sequences offer a new avenue for fold recognition. However, effectively comparing these predicted "maps" and scoring their similarity has remained a challenge. This paper addresses this by proposing and validating a new similarity metric, the congruence coefficient, specifically designed for comparing these structural maps.

Methods

The core of this work is the introduction and application of the congruence coefficient ($r_c$) as a similarity metric for protein maps. Unlike traditional correlation coefficients that measure deviations from the mean, the congruence coefficient measures deviations from zero, making it invariant to scaling and more suitable for comparing matrices that might have different baseline values.

Key methodological contributions include:

  • Mathematical Properties: The authors prove that the congruence coefficient has desirable mathematical properties, allowing for the efficient (polynomial time) computation of its exact mean and variance over all possible alignments between two matrices. This is crucial for assessing statistical significance.
  • Statistical Significance Assessment: They develop a method to rigorously assess the statistical significance of the congruence coefficient between two aligned maps, reframing it as a hypothesis test on the angle between unitary vectors. This helps determine if an observed similarity is likely due to chance.
  • Alignment-Aware Scoring: The congruence coefficient is used in conjunction with existing alignment tools. Instead of relying solely on the alignment tool's native scoring function, the congruence coefficient is calculated on the alignments produced by these tools. This allows for a more robust evaluation of template similarity.

Datasets

The study utilizes several benchmark datasets:

  • CASP12 and CASP13: Data from the Critical Assessment of Structure Prediction (CASP) experiments were used. Specifically, residue-residue contact predictions and simulated distance maps derived from structural predictions were employed.
  • ECOD Database: A large collection of over 27,000 non-homologous protein domain structures from the ECOD database served as templates. This database was pre-filtered to remove domains with high sequence identity to CASP targets, ensuring a challenging benchmark for detecting distant relationships.

Key Insights and Ideas

  • Congruence Coefficient's Suitability: The congruence coefficient's invariance to scaling and its ability to be rigorously assessed statistically make it a powerful tool for comparing structural maps, especially when dealing with potential noise or variations in predicted data.
  • Alignment Quality vs. Scoring Function: The paper highlights that while good alignment algorithms are important, the scoring function used to evaluate the similarity of aligned maps plays a critical role in fold recognition accuracy. The congruence coefficient often outperforms existing scoring functions.
  • Contact Maps vs. Distance Maps: The study suggests that contact map-based fold recognition, when using the congruence coefficient, can be more effective than distance map-based approaches, potentially due to the current state of prediction accuracy for each type of map.
  • Statistical Significance is Key: The ability to statistically assess the significance of a congruence coefficient score is crucial for filtering out spurious similarities and improving the reliability of fold recognition.

Key Results

  • Improved Fold Recognition Accuracy: Across both CASP12 and CASP13 datasets, using the congruence coefficient as a scoring function consistently improved fold recognition accuracy for all tested alignment tools (AlEigen, EigenThreader, Map_Align) compared to their native scoring functions.
  • Significant Gains with Contact Maps: For contact map-based fold recognition, the congruence coefficient led to substantial improvements, with True Positive Rates (TPR) often doubling or more for various ECOD hierarchy levels and top-k hit sets. For instance, with Map_Align+rc, TPRs reached up to 0.74 for top-20 hits on CASP13.
  • Comparable Performance with Distance Maps: For distance map-based fold recognition, the congruence coefficient showed comparable or slightly better performance than established metrics like TM-score and CE's Z-score.
  • Alignment Quality Analysis: The Z-score analysis revealed that while some alignment tools (like Map_Align) produced better alignments on average, the congruence coefficient's effectiveness was less dependent on the absolute quality of the alignment, suggesting its robustness.

Limitations and Future Work

  • Distance Map Prediction Quality: The performance of distance map-based fold recognition was noted to be significantly influenced by the quality of the underlying predicted structures. Improvements in structure prediction are expected to further boost distance map-based methods.
  • Limited FM Targets: While the study analyzed Free Modeling (FM) targets to assess performance on challenging cases, the limited number of such targets in the benchmark sets restricts drawing definitive conclusions.
  • Local vs. Global Alignment: The paper notes that structural alignment tools often perform local alignments, which might not be optimal for maximizing the congruence coefficient, suggesting that dedicated global distance map alignment tools could further enhance performance.
  • Computational Cost: While the mean and variance of the congruence coefficient can be computed efficiently, calculating the variance-covariance matrix for large matrices can be computationally intensive, though approximate methods exist.

The authors suggest that the congruence coefficient's effectiveness as a general similarity metric for protein map comparisons opens avenues for further research, particularly in developing more advanced distance map alignment tools.

Summary updated

Abstract

Abstract Motivation Protein fold recognition is a key step for template-based modeling approaches to protein structure prediction. Although closely related folds can be easily identified by sequence homology search in sequence databases, fold recognition is notoriously more difficult when it involves the identification of distantly related homologues. Recent progress in residue-residue contact and distance prediction opens up the possibility of improving fold recognition by using structural information contained in predicted distance and contact maps. Results Here we propose to use the congruence coefficient as a metric of similarity between maps. We prove that this metric has several interesting mathematical properties which allow one to compute in polynomial time its exact mean and variance over all possible (exponentially many) alignments between two symmetric matrices, and assess the statistical significance of similarity between aligned maps. We perform fold recognition tests by recovering predicted target contact/distance maps from the two most recent CASP editions and over 27,000 non-homologous structural templates from the ECOD database. On this large benchmark, we compare fold recognition performances of different alignment tools with their own similarity scores against those obtained using the congruence coefficient. We show that the congruence coefficient overall improves fold recognition over other methods, proving its effectiveness as a general similarity metric for protein map comparison. Availability The software CCpro is available as part of the Scratch suite http://scratch.proteomics.ics.uci.edu/
BibTeX
@misc{lena2020fold,
  author = {Pietro Di Lena and Pierre Baldi},
  title = {{Fold recognition by scoring protein map similarities using the congruence coefficient}},
  year = {2020},
  doi = {10.1101/2020.05.20.106484},
  howpublished = {bioRxiv},
}
Figure