TL;DR
This paper introduces SELECTpro, a novel structure-based method for selecting the most accurate protein model from a set of predicted structures. It achieves state-of-the-art performance by combining various energy terms and is robust to common issues in protein modeling, making it a valuable tool for the field.
What Problem Does This Paper Address?
Predicting the three-dimensional structure of a protein from its amino acid sequence is a fundamental challenge in computational biology. Once a set of potential protein structures (models) is generated, a crucial step is to identify the one that most closely resembles the actual, native structure. This is known as model selection or quality assessment. Existing methods fall into categories like evolutionary, consensus, and structure-based approaches, each with limitations. Consensus methods, for example, struggle when the native-like model is a structural outlier. Structure-based methods, while powerful, can be sensitive to local structural irregularities like steric clashes or chain breaks, which can skew their scoring. The authors aim to develop a structure-based method that is robust to these issues and can effectively select the best model from any set, regardless of size or redundancy.
Methods
SELECTpro is a structure-based model selection method that utilizes a novel energy function. This function is a linear combination of several terms, including:
- Predicted structural features: These are derived from external predictors for secondary structure (EPRED-SS), solvent accessibility (EPRED-ACC), and contact maps (EPRED-CM). These terms are novel and contribute to SELECTpro's ability to capture global structural properties.
- Physical terms: These include van der Waals forces (ELEN-JONES) and electrostatic interactions (EELECTRO), which are standard in protein energy functions.
- Statistical terms: These capture residue environment potentials (ESTAT-ENV) and context-independent (ESTAT-PW-CI) and context-dependent (ESTAT-PW-CD) pair-wise interactions, adapted from previous work.
- Novel terms: Two particularly innovative terms are EBETA (sheet formation) and ESC-HB (side-chain hydrogen bonding). EBETA specifically models the hydrogen bonding between beta-strand partners, accounting for realistic pairing and sheet formation without forcing strict register. ESC-HB penalizes unsatisfied hydrogen bonds for buried atoms.
The energy function is applied in two representations: a "reduced" representation focusing on backbone atoms and a "all-atom" representation that includes explicit side-chain atoms. The final score (EFINAL) is a combination of these. Parameter weights were optimized using a training set of protein domains and decoy models. The paper also introduces the "BLUNDER" metric to better assess model recovery by focusing on the worst model ranked above the true native structure, addressing limitations of simple ranking metrics.
Datasets
The primary datasets used for evaluation were from the CASP7 (Critical Assessment of protein Structure Prediction) experiment. This included:
- SetAll: All models submitted by participating groups for the CASP7 targets.
- SetComplete: A subset of SetAll containing only complete models (no missing backbone coordinates) and filtered for gross structural irregularities (e.g., Ca-Ca clashes, chain breaks, expanded termini). This dataset was designed to provide a more consistent basis for comparing structure-based methods.
Additionally, the paper evaluated SELECTpro on a benchmark set of 16 small proteins with large decoy sets (12,500 to 20,000 models each), previously used to benchmark the I-TASSER method.
Key Insights and Ideas
- Combining diverse energy terms is key: SELECTpro's success stems from integrating predicted structural features with physical and statistical potentials, rather than relying on a single type of scoring.
- Novelty in beta-strand and side-chain interactions: The EBETA and ESC-HB terms are significant contributions, addressing specific aspects of protein structure that are crucial for stability and often challenging for other methods.
- Robustness to model imperfections: The development of the BLUNDER metric and the careful curation of the SetComplete dataset highlight the authors' focus on creating a method that performs well even with imperfect or non-native-like models.
- Structure-based methods can be independent: The paper emphasizes that SELECTpro, by scoring models independently, can be easily integrated into existing protein structure prediction pipelines, offering broad applicability.
Key Results
- CASP7 Performance: SELECTpro achieved top results in the CASP7 model quality assessment category. It ranked the most native-like model first for 15 targets and within the top ten for 53 targets, outperforming other methods. The average GDT-TS difference between the top-ranked model and the true native model was 5.07.
- BLUNDER Metric Effectiveness: SELECTpro's AGDTBLUNDER% values were less than 10% for 40 targets and less than 20% for 63 targets on SetComplete, indicating strong recovery of near-native models.
- Large Decoy Set Comparison: On a benchmark set of 16 small proteins with large decoy sets, SELECTpro showed comparable performance to I-TASSER, a well-established consensus-based method, with an average GDT-TS of 63.7 compared to I-TASSER's 62.1.
- Pearson Correlation: SELECTpro showed a significant improvement in Pearson correlation between its scores and GDT-TS when moving from SetAll to SetComplete, demonstrating its robustness to incomplete and irregular models.
Limitations and Future Work
- Sensitivity to local irregularities: While SELECTpro is more resistant than some previous structure-based methods, it can still be affected by severe local structural problems like chain breaks or steric clashes, especially in the SetAll dataset. The authors address this by filtering for SetComplete.
- Error in model generation: The paper acknowledges an error in their use of Modeller for generating complete models from incomplete ones, which led to some outliers in the SetAll analysis. This was identified after the CASP7 competition.
- Potential for combination: The authors suggest that combining SELECTpro with consensus methods could lead to further improvements, citing examples where consensus methods excel and structure-based methods falter, and vice-versa.
- Availability: SELECTpro is made publicly available as a standalone application and a web server, encouraging its use and further development by the community.
Summary updated