A New Supervised Over-Sampling Algorithm with Application to Protein-Nucleotide Binding Residue Prediction

Jun Hu, Xue He, Dong‐Jun Yu, Xibei Yang, Jingyu Yang, Hong‐Bin ShenPLoS 2014 · DOI

Protein-nucleotide interactions are fundamental to biological processes, including cellular signaling and metabolism. Accurately identifying binding residues is essential for functional annotation and drug discovery. However, this task is a classic "needle in a haystack" problem: binding residues (the minority class) are significantly outnumbered by non-binding residues (the majority class). This severe class imbalance often leads machine learning models to become biased toward the majority class, resulting in poor predictive performance for the actual binding sites.

To address this, researchers have developed a novel Supervised Over-Sampling (SOS) algorithm and a corresponding predictor called TargetSOS. By synthesizing high-quality minority class samples, TargetSOS effectively alleviates class imbalance and achieves state-of-the-art performance in identifying binding residues across various nucleotide types.

The Supervised Over-Sampling (SOS) Approach

The core innovation of this study is the SOS algorithm, which moves beyond traditional over-sampling methods like SMOTE. While standard methods interpolate between existing samples, they can inadvertently introduce noise or create samples that lead to overfitting. SOS introduces a "quality control" mechanism using a classifier to vet synthetic data.

Feature Representation

Each protein residue is represented by a 391-dimensional vector incorporating:

  • LogisticPSSM: Evolutionary information derived from Position-Specific Scoring Matrices.
  • PSS: Predicted secondary structure information (e.g., alpha-helices or beta-strands).

The SOS Filtering Mechanism

  1. Initial Training: A Support Vector Machine (SVM) is trained on the original, imbalanced dataset.
  2. Synthetic Generation: New minority samples are generated.
  3. Confidence Filtering: The initial SVM predicts a confidence score for each new sample. A sample is only retained if its score falls within a specific "Goldilocks" interval ($T_{low}=0.6$ to $T_{high}=0.9$).

By excluding samples with near-perfect confidence scores ($>0.9$), the algorithm avoids adding redundant data that could cause overfitting. Instead, it focuses on challenging, diverse examples that force the model to learn more robust decision boundaries.

Experimental Results and Performance

The effectiveness of TargetSOS was evaluated using benchmark datasets, including ATP168 (168 ATP-binding proteins) and NUC5 (proteins binding to ATP, ADP, AMP, GTP, and GDP). The study utilized both five-fold cross-validation and independent validation tests to ensure the model's generalizability.

ROC curves comparing performance with and without the SOS algorithm on benchmark datasets.

Key findings include:

  • Significant Metric Gains: On the ATP168 dataset, the Matthews Correlation Coefficient (MCC)—a robust metric for imbalanced data—improved from 0.415 (without SOS) to 0.536 (with SOS).
  • Superiority Over Standard Samplers: SOS consistently outperformed Random Over-Sampling (ROS), SMOTE, and ADASYN across Sensitivity, Specificity, and Area Under the Curve (AUC) metrics.
  • State-of-the-Art Comparison: TargetSOS outperformed established predictors such as ATPint and NsitePred. For instance, in independent tests for GTP binding, TargetSOS achieved an impressive AUC of 0.900.

Conclusion and Future Directions

The SOS algorithm demonstrates that supervised data synthesis is a powerful tool for overcoming class imbalance in bioinformatics. While the current implementation requires manual tuning of confidence thresholds, its success in protein-nucleotide binding suggests broader applicability to other "rare event" prediction tasks, such as identifying protein-protein interaction sites or disulfide bonds. TargetSOS is available as a web-server, providing a valuable resource for the research community.

Summary updated

Abstract

Protein-nucleotide interactions are ubiquitous in a wide variety of biological processes. Accurately identifying interaction residues solely from protein sequences is useful for both protein function annotation and drug design, especially in the post-genomic era, as large volumes of protein data have not been functionally annotated. Protein-nucleotide binding residue prediction is a typical imbalanced learning problem, where binding residues are extremely fewer in number than non-binding residues. Alleviating the severity of class imbalance has been demonstrated to be a promising means of improving the prediction performance of a machine-learning-based predictor for class imbalance problems. However, little attention has been paid to the negative impact of class imbalance on protein-nucleotide binding residue prediction. In this study, we propose a new supervised over-sampling algorithm that synthesizes additional minority class samples to address class imbalance. The experimental results from protein-nucleotide interaction datasets demonstrate that the proposed supervised over-sampling algorithm can relieve the severity of class imbalance and help to improve prediction performance. Based on the proposed over-sampling algorithm, a predictor, called TargetSOS, is implemented for protein-nucleotide binding residue prediction. Cross-validation tests and independent validation tests demonstrate the effectiveness of TargetSOS. The web-server and datasets used in this study are freely available at http://www.csbio.sjtu.edu.cn/bioinf/TargetSOS/.
BibTeX
@inproceedings{hu2014new,
  author = {Jun Hu and Xue He and Dong‐Jun Yu and Xibei Yang and Jingyu Yang and Hong‐Bin Shen},
  title = {{A New Supervised Over-Sampling Algorithm with Application to Protein-Nucleotide Binding Residue Prediction}},
  booktitle = {PLoS},
  year = {2014},
  doi = {10.1371/journal.pone.0107676},
}
Figure