TL;DR
This paper introduces PreDisorder, a new web server for predicting intrinsically disordered regions in proteins using an ab initio sequence-based method. PreDisorder, based on the MULTICOM-CMFR algorithm, was rigorously benchmarked against 26 other predictors in the CASP8 competition and demonstrated competitive performance, making it a valuable tool for large-scale protein analysis.
What Problem Does This Paper Address?
Proteins are not always rigid, folded structures. Many proteins contain "intrinsically disordered regions" (IDRs) that lack a stable three-dimensional shape. While seemingly chaotic, these regions are crucial for protein function, enabling interactions with multiple partners and playing roles in various cellular pathways. However, IDRs can also hinder protein expression and purification, making them a challenge for experimental studies. Accurately predicting these regions is therefore vital for understanding protein structure, function, and for facilitating protein production for research. Current computational methods aim to identify these flexible segments directly from the amino acid sequence.
Methods
The core of PreDisorder is the MULTICOM-CMFR method, an ab initio approach utilizing 1-Dimensional Recursive Neural Networks (1D-RNNs). This method takes a protein sequence as input and processes it through several stages:
- Sequence Profiling: The input sequence is aligned against existing protein profiles using PSI-BLAST to generate a profile representing the sequence's characteristics.
- Feature Engineering: This profile is combined with predicted secondary structure (helix, strand, coil) and solvent accessibility information for each residue. This creates a vector of 25 values for each amino acid in the sequence.
- 1D-RNN Prediction: The engineered features are fed into a 1D-RNN, which outputs a probability for each residue being disordered.
- Rescaling: The predicted probabilities are then rescaled to better match the observed ratio of disordered residues in the training data.
Additionally, the authors developed a meta-predictor (MULTICOM), which combines predictions from multiple other disorder predictors. This meta-predictor works by removing a few inaccurate predictors and then averaging the outputs of the remaining ones, a simpler approach than consensus voting.
Datasets
The primary dataset used for evaluation was the CASP8 dataset, which included 117 protein targets for which experimental structures were available. These targets comprised a total of 25,431 residues. The disorder annotations for these targets were curated by Dr. McGuffin. The evaluation was further broken down into two subsets: 97 X-ray structures and 20 NMR structures.
Key Insights and Ideas
- Ab initio methods are competitive: The study highlights that ab initio methods, like MULTICOM-CMFR, can achieve performance comparable to more complex meta or clustering methods, especially when well-designed. This is significant because ab initio methods can predict disorder solely from sequence, without needing information from other predictors or structural templates.
- Meta-prediction offers robustness: While MULTICOM-CMFR performs well, the MULTICOM meta-predictor demonstrates the power of combining multiple predictions for improved accuracy and reliability.
- CASP8 as a crucial benchmark: The paper underscores the importance of large-scale, blind assessments like CASP8 for objectively evaluating and advancing computational biology tools.
- Disorder prediction is nearing its limit: The authors suggest that the accuracy of current disorder prediction methods, as evidenced by high AUC scores on X-ray targets, might be approaching the inherent limitations of experimental data and prediction capabilities.
Key Results
- PreDisorder (using MULTICOM-CMFR) was benchmarked against 26 other disorder predictors on the CASP8 dataset.
- MULTICOM-CMFR performed favorably compared to other ab initio methods and was comparable to the best meta and clustering methods.
- The MULTICOM meta-predictor also showed strong performance.
- The paper provides detailed performance metrics (ROC scores, sensitivity, specificity, F-measure) for all evaluated predictors in Tables 1, 2, and 3, showcasing PreDisorder's competitive standing. For instance, on the combined 117 targets, MULTICOM achieved an ROC score of 0.879, and MULTICOM-CMFR achieved 0.859, placing them among the top predictors.
Limitations and Future Work
- Limitations: The paper notes that no single method performed decisively better than all others across all measures. The high performance in negative specificity and negative sensitivity is attributed to the fact that most residues in a protein are ordered, making it easier to correctly identify them as such.
- Future Work: The PreDisorder server is presented as a "fast and reliable server which can be used to predict protein disordered regions on genomic scale." This implies a future direction of applying the tool for large-scale annotation and analysis of protein disorder across entire proteomes. The authors also suggest that further improvements could be made by exploring more sophisticated meta-prediction strategies or by incorporating additional sequence-derived features.
Summary updated