Abstract
Motivated by applications in protein function prediction, we consider a challenging supervised classification setting in which positive labels are scarce and there are no explicit negative labels. The learning algorithm must thus select which unlabeled examples to use as negative training points, possibly ending up with an unbalanced learning problem. We address these issues by proposing an algorithm that combines active learning (for selecting negative examples) with imbalance-aware learning (for mitigating the label imbalance). In our experiments we observe that these two techniques operate synergistically, outperforming state-of-the-art methods on standard protein function prediction benchmarks.
Introduction
In the field of bioinformatics, protein function prediction is often hindered by the lack of explicit negative labels. In databases like Gene Ontology (GO), a protein not annotated with a specific function might be a true negative or simply a function yet to be discovered. This creates a "Positive and Unlabeled" (PU) learning scenario characterized by two main difficulties:
- Noisy Negatives: Treating all unlabeled data as negative introduces noise from undiscovered positives.
- Extreme Imbalance: Confirmed positives are often vastly outnumbered by the unlabeled majority, causing standard algorithms to ignore the minority class.
Methods
The authors propose a two-phase approach that treats negative selection as an active process:
- Active Negative Selection: Using Active Learning (AL), the model iteratively identifies the "most informative" unlabeled examples—those near the decision boundary where the model is least confident—to serve as the negative training set.
- Imbalance-aware Classification: The final model is trained using techniques designed for lopsided data. Two variants were tested: SVM AL (cost-sensitive Support Vector Machine) and RF AL (Balanced Random Forest).
The core innovation is the synergy between these phases: AL identifies the most useful negatives, while cost-sensitivity ensures the scarce positives are not overwhelmed.
Key Results
The framework was evaluated on Saccharomyces cerevisiae and Homo sapiens datasets using protein-protein interaction networks from the STRING database. Temporal holdouts were used to simulate real-world discovery.
Performance and Synergy
The SVM AL method significantly outperformed eight state-of-the-art baselines, including GBA and COSNet ($p < 0.001$). The researchers found that the combination of negative selection and imbalance handling was essential; neither technique provided significant gains in isolation.
Efficiency and Scalability
As illustrated in the figure above, the SVM-based approach offers a substantial advantage in computational efficiency. While both SVM and RF maintain stable F-measures across different step sizes ($s$), the time required for RF AL is significantly higher. For example, SVM AL typically completes cycles in 50–100 seconds, whereas RF AL can exceed 400 seconds in some configurations. This makes SVM AL the more practical choice for large-scale genomic studies.
Conclusion
This research demonstrates that a "flat" classification approach, when combined with intelligent negative selection and cost-sensitivity, can outperform complex hierarchical methods in protein function prediction. Future work will focus on tuning parameter sensitivity and potentially integrating hierarchical GO information to further enhance predictive accuracy.
Summary updated