IDP–CRF: Intrinsically Disordered Protein/Region Identification Based on Conditional Random Fields

Yumeng Liu, Xiaolong Wang, Bin LiuInternational Journal of Molecular Sciences 2018 · DOI

TL;DR

This paper introduces IDP-CRF, a novel computational method for identifying intrinsically disordered proteins and regions (IDPs/IDRs) by leveraging Conditional Random Fields (CRFs) and a comprehensive set of sequence-based features. IDP-CRF significantly outperforms existing state-of-the-art methods, offering a more accurate and efficient tool for protein sequence analysis.

What Problem Does This Paper Address?

Intrinsically disordered proteins and regions (IDPs/IDRs) lack a stable 3D structure in their native state, yet play crucial roles in various biological functions and are implicated in diseases. Accurately identifying these regions is vital for understanding biological mechanisms and disease pathways. While experimental methods exist, they are time-consuming and expensive, necessitating efficient computational approaches. Existing computational methods often struggle to effectively capture the "sequence-order effect" – the fact that disordered regions exhibit global sequence patterns and that adjacent residues tend to share similar characteristics. This limitation hinders the development of highly accurate predictors.

Methods

The core of IDP-CRF lies in its application of Conditional Random Fields (CRFs), a sequence labeling model that excels at capturing dependencies between adjacent elements in a sequence. Unlike traditional classification models that treat each amino acid residue independently, CRFs can model the relationships between neighboring residues, thus better accounting for the global sequence patterns characteristic of IDPs/IDRs.

IDP-CRF incorporates a rich set of sequence-based features to represent each amino acid residue:

  • Position-Specific Scoring Matrices (PSSMs): These capture evolutionary information, indicating the likelihood of each amino acid at a specific position based on alignments with homologous sequences.
  • Kmer: This represents the occurrence frequencies of short amino acid subsequences (k-mers) within a local window, providing a direct representation of the protein sequence.
  • Predicted Secondary Structure: Information about whether a residue is predicted to be in a helix, beta strand, or coil conformation.
  • Relative Solvent Accessibility: The predicted accessibility of a residue to the solvent, which can be indicative of its structural state.

The authors also highlight the use of a transition feature that explicitly models the probability of transitioning between ordered and disordered states for adjacent residues.

Datasets

The study utilizes a newly constructed, comprehensive, and updated benchmark dataset (S). This dataset is a union of:

  • S1: Proteins from the MobiDB database with structures solved by X-ray crystallography, where missing electron densities are considered IDPs/IDRs. Proteins were filtered based on resolution (≤ 2Å), length (≥ 30 residues), and the presence of at least one IDR.
  • S2: Confident annotations from the DisProt_v7.0 database.

Redundant proteins were removed using the Blastclust algorithm with a 25% sequence identity cutoff, resulting in 5273 proteins.

For independent evaluation, the study uses two widely recognized benchmark independent datasets: MxD494 and SL329.

Key Insights and Ideas

  • CRFs are well-suited for capturing sequence context: The paper strongly advocates for sequence labeling models like CRFs over independent classification approaches for IDP/IDR prediction, demonstrating that explicitly modeling the relationships between adjacent residues is crucial for capturing global disorder patterns.
  • Feature richness matters: The combination of PSSMs, kmer, secondary structure, and solvent accessibility provides a more comprehensive representation of protein sequence characteristics, leading to improved prediction accuracy.
  • Dataset quality is paramount: Training on a larger, more up-to-date, and carefully curated benchmark dataset (combining MobiDB and DisProt) significantly enhances the predictor's generalization ability and performance.
  • Balanced training data is important: The study shows that the performance of IDP-CRF and other predictors is optimized when the ratio of positive (disordered) to negative (ordered) samples is around 1:2, suggesting that careful handling of imbalanced datasets is necessary.

Key Results

  • Superior performance: IDP-CRF consistently outperformed 25 existing state-of-the-art methods on both the benchmark and independent test datasets.
  • High accuracy metrics: On the benchmark dataset, IDP-CRF achieved the highest accuracy (ACC), sensitivity (Sn), specificity (Sp), and Matthew's correlation coefficient (MCC) compared to RF, SVM, and ANN predictors.
  • Continuous vs. Discontinuous Predictions: Visualizations of predictions on example proteins showed that IDP-CRF tends to predict disordered regions more continuously, aligning better with actual disordered regions, whereas classification-based predictors often produced fragmented predictions.
  • Reduced false positives: IDP-CRF also demonstrated a lower number of false positives (predicted disordered residues in ordered regions) compared to classification-based methods.
  • Comparable to top-tier methods: On the MxD494 dataset, IDP-CRF showed comparable performance to the meta-predictor MFDp and outperformed other related methods. On the SL329 dataset, it was highly comparable to SPOT-disorder.

Limitations and Future Work

  • Future exploration of ML algorithms: The authors express interest in exploring new machine learning algorithms to further enhance prediction accuracy.
  • Computational cost: While not explicitly detailed as a limitation for IDP-CRF, the paper notes that some existing methods (like SPOT-disorder) have high computational costs, implying that efficiency is a consideration. IDP-CRF's use of CRFs and comprehensive features might imply a trade-off between accuracy and computational speed, though it is presented as an improvement over previous CRF methods.

Summary updated

Abstract

Accurate prediction of intrinsically disordered proteins/regions is one of the most important tasks in bioinformatics, and some computational predictors have been proposed to solve this problem. How to efficiently incorporate the sequence-order effect is critical for constructing an accurate predictor because disordered region distributions show global sequence patterns. In order to capture these sequence patterns, several sequence labelling models have been applied to this field, such as conditional random fields (CRFs). However, these methods suffer from certain disadvantages. In this study, we proposed a new computational predictor called IDP⁻CRF, which is trained on an updated benchmark dataset based on the MobiDB database and the DisProt database, and incorporates more comprehensive sequence-based features, including PSSMs (position-specific scoring matrices), kmer, predicted secondary structures, and relative solvent accessibilities. Experimental results on the benchmark dataset and two independent datasets show that IDP⁻CRF outperforms 25 existing state-of-the-art methods in this field, demonstrating that IDP⁻CRF is a very useful tool for identifying IDPs/IDRs (intrinsically disordered proteins/regions). We anticipate that IDP⁻CRF will facilitate the development of protein sequence analysis.
BibTeX
@article{liu2018idpcrf,
  author = {Yumeng Liu and Xiaolong Wang and Bin Liu},
  title = {{IDP–CRF: Intrinsically Disordered Protein/Region Identification Based on Conditional Random Fields}},
  journal = {International Journal of Molecular Sciences},
  year = {2018},
  doi = {10.3390/ijms19092483},
}
Figure