An Active Transfer Learning Framework for Protein-Protein Interaction Extraction

Lishuang Li, Xinyu He, Jieqiong Zheng, Degen Huang, Fuji Ren

Abstract

Extracting protein-protein interactions (PPI) from biomedical literature is a critical task for biological knowledge discovery, yet it is hampered by the high cost of expert data labeling. This paper presents ActTrAdaBoost, a novel framework that integrates transfer learning with active learning. By employing a "relative distribution" weighting method (RDTrAdaBoost), the framework effectively mitigates negative transfer from source domains. Experimental results on gold-standard corpora demonstrate that ActTrAdaBoost significantly outperforms traditional supervised and transfer learning methods, particularly when labeled target data is extremely scarce.

Introduction

The rapid growth of biomedical literature necessitates automated methods for Protein-Protein Interaction Extraction (PPIE). While supervised machine learning models have shown promise, their performance depends on large volumes of high-quality labeled data. Transfer learning offers a potential solution by leveraging existing labeled datasets from related domains. However, differences in data distributions between source and target domains often lead to "negative transfer," where the source data degrades model performance. The authors aim to bridge this gap by making transfer learning more robust and data-efficient.

The ActTrAdaBoost Framework

To address these challenges, the authors propose the ActTrAdaBoost framework, which combines an improved transfer learning algorithm with an active learning loop.

The ActTrAdaBoost framework architecture, illustrating the integration of RDTrAdaBoost and active learning.

The framework consists of two primary components:

  1. RDTrAdaBoost (Relative Distribution TrAdaBoost): Standard TrAdaBoost treats all source instances equally at initialization. RDTrAdaBoost improves upon this by initializing weights based on the relative distribution ($P_T(x)/P_S(x)$). This approach prioritizes source instances that closely resemble the target domain, filtering out noise and reducing the risk of negative transfer.
  2. Active Learning Integration: The framework incorporates an active learning loop using a Support Vector Machine (SVM) to identify samples with high uncertainty. When the model's confidence is low or the training set is insufficient, it queries a human expert for labels. For high-confidence predictions, the model uses its own output to expand the training set autonomously.

Experimental Evaluation

The framework was evaluated using three benchmark biomedical corpora: AIMed, IEPA, and HPRD50. The authors utilized Kullback–Leibler (KL) divergence to confirm significant distributional differences between these datasets, justifying the need for a robust transfer learning approach.

Results and Discussion

ActTrAdaBoost consistently outperformed baseline models, including standard SVMs and the original TrAdaBoost algorithm. The framework's strengths are most apparent in data-sparse scenarios.

Performance comparison (F-score) across different proportions of expert-labeled examples, highlighting the significant gains of ActTrAdaBoost at low labeling rates.

Key findings include:

  • Efficiency at the Edge: With only 2% of the target data labeled by experts, ActTrAdaBoost achieved a 26.69% improvement in F-score over standard TrAdaBoost on the IEPA dataset.
  • Robustness to Negative Transfer: RDTrAdaBoost successfully maintained performance even when source and target domains were highly dissimilar (e.g., using HPRD50 as a source for IEPA), where traditional methods typically falter.

Conclusion

ActTrAdaBoost provides a powerful solution for PPIE in scenarios where expert annotation is limited. By combining distribution-aware transfer learning with active learning, the framework maximizes the utility of both existing source data and limited target labels. Future work may involve integrating deep learning architectures to automate feature extraction and further enhance performance.

Summary updated

Abstract

Protein-Protein Interaction Extraction (PPIE) from biomedical literatures is an important task in biomedical text mining and has achieved great success on public datasets. However, in real-world applications, the existing PPI extraction methods are limited to label effort. Therefore, transfer learning method is applied to reduce the cost of manual labeling. Current transfer learning methods suffer from negative transfer and lower performance. To tackle this problem, an improved TrAdaBoost algorithm is proposed, that is, relative distribution is introduced to initialize the weights of TrAdaBoost to overcome the negative transfer caused by domain differences. To make further improvement on the performance of transfer learning, an approach combining active learning with the improved TrAdaBoost is presented. The experimental results on publicly available PPI corpora show that our method outperforms TrAdaBoost and SVM when the labeled data is insufficient,and on document classification corpora, it also illustrates that the proposed approaches can achieve better performance than TrAdaBoost and TPTSVM in final, which verifies the effectiveness of our methods.
BibTeX
@article{li2018active,
  author = {Lishuang Li and Xinyu He and Jieqiong Zheng and Degen Huang and Fuji Ren},
  title = {{An Active Transfer Learning Framework for Protein-Protein Interaction Extraction}},
  journal = {IEICE Trans. Information and Systems},
  year = {2018},
  doi = {10.1587/transinf.2017edp7232},
}
Figure