TL;DR
This paper investigates how well predicted protein structures from the CASP competition can be used to infer biological function. They found that while overall structural quality generally correlates with functional utility, the best-ranked global predictions don't always have the best local functional features, highlighting the need for specialized tools to assess functional relevance.
What Problem Does This Paper Address?
Predicting the 3D structure of proteins from their amino acid sequence has been a long-standing challenge in biology. While significant progress has been made, particularly with the advent of deep learning methods, the ultimate goal is to use these predicted structures to understand protein function. However, it's not always straightforward to quantify how "useful" a predicted structure is for inferring function. This paper tackles the question: how well do the predicted protein structures from the Critical Assessment of protein Structure Prediction (CASP) competition, when compared to experimentally determined structures, actually help us understand a protein's biological role? This is crucial because functional annotation is a key step in drug discovery, understanding disease mechanisms, and designing novel proteins.
Methods
The core of this study relies on a computational tool called PocketFEATURE (PF), which was previously developed by the authors. PF is designed to represent and compare the "microenvironments" of functional sites within proteins. A microenvironment is essentially a local, spherical region around a key residue or a functional center, capturing its physicochemical and structural features.
The authors applied PF to compare the microenvironments of specific functional sites in predicted protein structures against those in experimentally solved structures. They categorized these functional sites into three types:
- Holo sites: Pockets where a ligand is known to bind in the experimental structure.
- Apo sites: Pockets that are predicted or suggested to bind ligands, based on experimental authors' notes or computational predictions.
- Critical patches: Regions containing important motifs, loops, or residues associated with disease mutations.
They then compared the PF-based functional assessments with the official CASP assessments of overall structural quality (like GDT, TM, and RMSD). They performed two types of analyses: "all-models" (averaging scores across all predictions from a server) and "model-1" (focusing only on the top-ranked prediction from each server).
Datasets
The study utilized data from the CASP12 competition, specifically focusing on 25 protein targets for which experimental structural information and functional annotations were available. For these targets, they analyzed a total of 28 functional sites, categorized as described above. They also drew upon existing mutation databases (ClinVar and HGMD) for their analysis of missense mutation sites.
Key Insights and Ideas
- Local vs. Global Quality: The study highlights a crucial distinction: a protein structure that is globally accurate (high GDT score) doesn't necessarily mean its functional sites are accurately represented. Conversely, a prediction with a lower global score might still capture the local functional environment well.
- Holo vs. Apo Site Prediction: Prediction methods generally perform better on "apo sites" (sites without a bound ligand in the experimental structure) than on "holo sites" (sites with a bound ligand). This is likely because the presence of a ligand significantly alters the physicochemical environment of the binding pocket, making it harder to predict accurately without explicit ligand information.
- Functional Sites and Protein-Protein Interactions: The research suggests that many functional sites are involved in protein-protein interactions, a complex area of prediction that is heavily influenced by template choice.
- The Power of Microenvironments: The PF tool demonstrates that focusing on the detailed physicochemical properties of local microenvironments, rather than just overall structural similarity, can provide a more nuanced assessment of functional relevance.
Key Results
- Correlation between Structure and Function: Overall structural quality (measured by CASP metrics) generally correlates with functional utility, especially for the top-ranked "model-1" predictions.
- Performance Differences: The correlation between functional assessments and CASP rankings was consistently higher for apo sites and critical patches compared to holo sites.
- Server Performance: Certain servers showed consistently good performance for specific types of functional sites, indicating that specialized prediction methods might be beneficial.
- Case Study Example (T0891): For a heme-binding protein (T0891), the best-ranked functional model (PF-zscore -2.035) captured more similar microenvironments around the heme-binding site than the best structural model (GDT score 91.74), even though the latter had a higher GDT score. This exemplifies how functional assessment can reveal strengths missed by purely structural metrics.
- Mutation Site Analysis: For missense mutation sites, the study found that local prediction accuracy of solvent accessibility or secondary structure did not strongly correlate with the overall structural quality of the prediction, suggesting that predicting the pathogenicity of a variant is a complex task that might require more than just accurate structure prediction.
Limitations and Future Work
- Site Definition Ambiguity: The authors acknowledge that defining functional sites and their centers can be challenging, even with input from experimentalists. This ambiguity is a significant limitation for functional assessment methods. They suggest a more structured approach for retrieving biological relevance from experimental contributors in future CASP assessments.
- Template Bias in Assemblies: For protein assemblies, the quality of the predicted features was heavily influenced by the choice of templates, highlighting the difficulty in predicting functions arising from multi-subunit interactions.
- Future Directions: The paper implicitly suggests that developing more specialized tools for functional site assessment, beyond general structural quality metrics, is crucial. Further research could focus on improving the prediction of protein-protein interfaces and developing more robust methods for predicting the functional impact of mutations.
Summary updated