Leveraging Prior Knowledge for Protein-Protein Interaction Extraction with Memory Network

Huiwei Zhou, Zhuang Liu, Shixian Ning, Yunlong Yang, Chengkun Lang, Yingyu Lin, Kun Ma

Introduction

Understanding protein-protein interactions (PPI) is fundamental to precision medicine, particularly in identifying how genetic mutations disrupt cellular pathways. However, the exponential growth of biomedical literature makes manual curation increasingly difficult. While automated Natural Language Processing (NLP) tools have been developed, extracting specific "mutated" interactions (PPIm) remains challenging due to contextual complexity in long sentences and the failure of existing models to leverage structured data from established Knowledge Bases (KBs).

To address these issues, researchers have developed a novel Memory Network-based Model (MNM). This approach treats relation extraction as a fusion of textual context and prior knowledge, significantly improving performance on the BioCreative VI Track 4 PPI extraction task.

Methodology

The MNM architecture is designed to integrate structured prior knowledge with deep learning attention mechanisms.

Architecture of the Memory Network-based Model (MNM) showing the integration of entity embeddings and context word embeddings through attention layers.

Knowledge Integration

The model utilizes TransE to convert KB "triples" (Head Entity, Relation, Tail Entity) into continuous vector spaces. This provides a structural anchor, giving the model a "pre-existing" understanding of protein relationships before processing any text.

Dual Memory Networks

The core architecture features two parallel memory networks, one for each protein in a candidate pair. These networks share parameters, enabling the model to learn mutual influences. Instead of linear processing, the memory network uses an attention mechanism to score context words based on their relevance to the specific protein pair.

Multi-Layered Refinement

Unlike standard LSTMs with local memory cells, the MNM stacks multiple computational layers over an external memory. This allows the model to refine its focus over several passes, effectively capturing global dependencies that are often lost in complex biomedical sentences.

Experimental Setup

The model was evaluated using the BioCreative VI Track 4 PPI extraction corpus, consisting of 597 training and 1,500 test PubMed abstracts. Prior knowledge was sourced from a merged dataset of over 1.5 million triples and 84,000 protein entities from IntAct and BioGrid.

Entity and triple coverage across training and test sets relative to existing Knowledge Bases (KBs).

As shown in the coverage analysis, a significant portion of entities in both sets are already present in KBs, though triple coverage is lower, particularly in the test set. This highlights the importance of a model that can generalize from known entities to new, potentially unrecorded interactions.

Results and Discussion

The MNM model, combined with simple heuristic post-processing rules, achieved a new state-of-the-art F1-score of 37.48% on the BioCreative VI Track 4 task. This outperformed the previous top-ranked rule-based systems (33.94%) and standard deep learning approaches.

Key Findings

  • Impact of Prior Knowledge: Incorporating TransE-based entity embeddings improved the F1-score by 1.94% compared to models using only averaged word embeddings.
  • Memory Efficiency: External memory proved superior to the "locked" hidden states of LSTMs for long-sequence biomedical text. The ability to perform multiple "reads" and "writes" allows for better global information capture.
  • Optimal Depth: Performance peaked at four computational layers. Beyond this, gradient vanishing issues hindered training effectiveness.

Conclusion and Future Work

While the MNM represents a significant advancement, several challenges remain. A primary bottleneck is the recall of initial protein tagging; approximately 44.42% of relations were lost due to unrecognized entities by the GNormPlus tagger. Future work will focus on moving from sentence-level to document-level extraction using hierarchical attention networks and improving the model's robustness to noise or contradictions within Knowledge Bases.

Summary updated

BibTeX
@article{zhou2018leveraging,
  author = {Huiwei Zhou and Zhuang Liu and Shixian Ning and Yunlong Yang and Chengkun Lang and Yingyu Lin and Kun Ma},
  title = {{Leveraging Prior Knowledge for Protein-Protein Interaction Extraction with Memory Network}},
  journal = {Database J. Biol. Databases Curation},
  year = {2018},
  doi = {10.1093/database/bay071},
}
Figure