Deep learning for extracting protein-protein interactions from biomedical literature

Yifan Peng, Zhiyong Lu

TL;DR

This paper introduces McDepCNN, a novel deep learning model that leverages dependency parse information to improve the extraction of protein-protein interactions (PPIs) from biomedical literature. By incorporating syntactic structure as a separate "channel" in a convolutional neural network, McDepCNN achieves state-of-the-art performance and demonstrates superior generalizability across different datasets.

What Problem Does This Paper Address?

The biomedical literature is a vast repository of information, and a critical task is to automatically identify protein-protein interactions (PPIs). These interactions are fundamental to understanding biological processes. While traditional machine learning methods have been used for this, they often rely on manual feature engineering, which is time-consuming and requires expert knowledge. More recent deep learning approaches, particularly Convolutional Neural Networks (CNNs), have shown promise but struggle to effectively incorporate rich linguistic and semantic information, often leading to suboptimal performance. The key challenge is how to effectively integrate the complex syntactic structure of sentences into deep learning models for PPI extraction.

Methods

The authors propose McDepCNN, a multichannel dependency-based convolutional neural network. The core innovation lies in how it processes sentence information:

  • Multichannel Input: Instead of a single input stream, McDepCNN uses multiple "channels" to represent each word.
  • Word Channel: This is the standard input, combining word embeddings with part-of-speech, chunk, named entity, and positional features.
  • Head Word Channel: This is the novel contribution. It uses the dependency "head" of each word as a separate channel. This intuitively captures the syntactic structure of the sentence, where words are linked to their modifiers or governing words. This channel effectively breaks down the dependency graph into structural pairs, providing a richer representation than just the shortest path between entities.
  • Convolutional Neural Network (CNN): The model then applies convolutional layers with multiple window sizes to these channels to extract local features.
  • Max Pooling: Global features are extracted by taking the maximum value from each feature map, aiming to capture the most salient information.
  • Fully Connected Layer with Softmax: Finally, a fully connected layer and a softmax function are used for binary classification (PPI or no PPI).

Datasets

The paper evaluates McDepCNN on two publicly available benchmarking PPI corpora:

  • AIMed: Contains 1,955 sentences with 1,000 positive PPI instances.
  • BioInfer: Contains 1,100 sentences with 2,534 positive PPI instances.

These corpora are standard for PPI extraction research. The authors also refer to a subset of "difficult" instances previously identified in the literature for further analysis.

Key Insights and Ideas

  • Syntactic Structure as a Feature: The paper demonstrates that explicitly encoding syntactic dependency information as a separate input channel in a CNN can significantly boost performance. This moves beyond simply using dependency paths as features and instead integrates the structural relationships directly into the learning process.
  • Generalizability: The "head word channel" appears to capture longer-range dependencies and contextual information that might be missed by traditional fixed-size windows or shallow linguistic models. This leads to better performance on cross-corpus evaluations, suggesting the model is more robust and less prone to overfitting to specific dataset characteristics.
  • Beyond Shortest Path: Unlike some prior CNN approaches that focused solely on the shortest path between interacting proteins, McDepCNN considers the broader sentence structure, which is shown to be beneficial.
  • "Difficult" Instances: The model shows a remarkable improvement on instances that are hard for traditional kernel-based methods, highlighting its ability to handle more complex sentence structures and relationships.

Key Results

  • State-of-the-Art Performance: McDepCNN achieved the highest precision and F1-score on both the AIMed and BioInfer datasets compared to existing feature-based and single-kernel methods.
  • Cross-Corpus Improvement: The model achieved a 24.4% relative improvement in F1-score on cross-corpus evaluation (trained on BioInfer, tested on AIMed) and 18.2% vice versa, compared to a shallow linguistic model.
  • Difficult Instances: McDepCNN showed a more than three-fold improvement in F1-score on difficult PPI instances compared to kernel-based methods.
  • Contribution of Components: Ablation studies showed that using a single window size of 3 was optimal, and removing the "head word channel" (reducing it to a single channel) resulted in a performance drop of 1.1%.

Limitations and Future Work

  • Limitations: The paper acknowledges that while their model performs well, it does not directly compare with some recent deep learning approaches that artificially filtered sentences. They also note that their results on difficult instances are compared against kernel methods using different datasets, which might limit direct comparability.
  • Future Work: The authors suggest several avenues for future research:
  • Integrating Diverse Resources: Exploring how to combine various resources, similar to rich-feature-based methods, to further enhance performance.
  • Extending to Cross-Sentence PPIs: Adapting the model to identify PPIs that span across multiple sentences or even documents.
  • Generalizing to Other Biomedical Relations: Applying the McDepCNN framework to extract other types of biomedical relations, such as chemical-disease interactions.

Summary updated

Abstract

State-of-the-art methods for protein-protein interaction (PPI) extraction are primarily feature-based or kernel-based by leveraging lexical and syntactic information. But how to incorporate such knowledge in the recent deep learning methods remains an open question. In this paper, we propose a multichannel dependency-based convolutional neural network model (McDepCNN). It applies one channel to the embedding vector of each word in the sentence, and another channel to the embedding vector of the head of the corresponding word. Therefore, the model can use richer information obtained from different channels. Experiments on two public benchmarking datasets, AIMed and BioInfer, demonstrate that McDepCNN compares favorably to the state-of-the-art rich-feature and single-kernel based methods. In addition, McDepCNN achieves 24.4% relative improvement in F1-score over the state-of-the-art methods on cross-corpus evaluation and 12% improvement in F1-score over kernel-based methods on "difficult" instances. These results suggest that McDepCNN generalizes more easily over different corpora, and is capable of capturing long distance features in the sentences.
BibTeX
@inproceedings{peng2017deep,
  author = {Yifan Peng and Zhiyong Lu},
  title = {{Deep learning for extracting protein-protein interactions from biomedical literature}},
  booktitle = {BioNLP},
  year = {2017},
  url = {https://openreview.net/forum?id=Lr_2jjU2m_},
}
Figure