Joint Models for Extracting Adverse Drug Events from Biomedical Text

Fei Li, Yue Zhang, Meishan Zhang, Donghong JiIJCAI 2016 · OpenReview

Abstract

Extracting adverse drug events (ADEs) is a critical task in the biomedical community, traditionally handled by pipeline models that first recognize drug and disease entities and then identify relationships between them. However, pipeline models suffer from error propagation and a lack of information integration between subtasks. In this paper, we investigate joint models for the simultaneous extraction of drugs, diseases, and ADEs. We compare a discrete transition-based model with a deep neural variant. Experimental results on a standard ADE corpus demonstrate that joint modeling significantly outperforms state-of-the-art pipeline baselines. Furthermore, the integration of neural features significantly improves recall, leading to a more robust extraction system.

Introduction

Monitoring drug safety, or pharmacovigilance, relies heavily on the accurate extraction of Adverse Drug Events (ADEs) from the vast body of medical literature. Traditionally, this has been approached as a two-step "pipeline": first, Named Entity Recognition (NER) identifies mentions of drugs and diseases; second, Relation Extraction determines if a specific drug-disease pair represents an ADE (e.g., "Gliclazide-induced hepatitis").

The primary limitation of this approach is error propagation. If the NER step fails to identify a disease, the subsequent relation extraction step cannot recover that information. Moreover, these tasks are inherently related; the context that suggests an ADE relationship often provides strong cues for identifying the entities themselves. Standard pipelines cannot leverage this bidirectional information flow. This research proposes joint models to address these flaws by solving both tasks simultaneously.

Methodology

Transition-Based Framework

The authors employ a transition-based model that processes sentences incrementally from left to right. Similar to a shift-reduce parser, the model uses a series of actions to build the output. Actions include labeling words (e.g., BC for "Beginning of Drug," BD for "Beginning of Disease") and specific relation actions (Y or N) to confirm if a newly identified entity has an ADE relationship with previously identified entities in the sentence.

Discrete vs. Neural Joint Models

Two variants of the joint model were developed:

  1. Discrete Joint Model: This version utilizes a maximum-entropy classifier with hand-crafted features. It incorporates "local" features (words, POS tags) and "global" features that capture complex interactions, such as coordinating structures (e.g., "Drug A and Drug B caused Disease C").
  2. Neural Joint Model: To address feature sparsity and the "unseen word" problem, a neural architecture was implemented. It uses a Convolutional Neural Network (CNN) to transform variable-length entity mentions into fixed-length vectors. These vectors are then processed by a feed-forward network to predict the next transition action.
Architecture of the Neural Joint Model, utilizing CNNs for variable-length feature extraction.

Experimental Evaluation

The models were evaluated on the ADE corpus, a standard benchmark consisting of 1,644 PubMed abstracts containing approximately 6,821 sentences with at least one ADE. The researchers utilized 10-fold cross-validation to compare their joint models against a state-of-the-art pipeline baseline and previous joint modeling work (Li et al., 2015).

Results and Discussion

The experimental results confirm that joint modeling is superior to pipeline approaches. Even a basic discrete joint model using only local features outperformed the baseline, validating the hypothesis that reducing error propagation is a primary driver of success.

Performance comparison (P, R, F1) for Entity Recognition and ADE Extraction across different model configurations.

As shown in the performance data, the addition of global features to the discrete model provided a substantial boost to the ADE extraction F1-score, rising from 56.0% to 61.1%. However, the Neural Joint Model achieved the highest overall performance, reaching an F1-score of 63.4% for ADE extraction.

A key insight from the error analysis is the neural model's superior recall. By using dense embeddings, the neural model generalizes better to synonyms and ambiguous contexts that hand-crafted patterns might miss, resulting in significantly fewer false negatives. The model is particularly effective at handling coordinating relations where multiple drugs are linked to the same adverse effect.

Conclusion

This study demonstrates that joint models effectively mitigate the weaknesses of traditional pipeline methods in biomedical relation extraction. While the current implementation uses greedy decoding for efficiency, future work may explore beam search or exact inference to further refine accuracy. Additionally, incorporating syntactic dependency trees into the neural architecture could help capture long-distance relationships between entities, further advancing the state-of-the-art in pharmacovigilance.

Summary updated

Abstract

Extracting adverse drug events receives much research attention in the biomedical community. Previous work adopts pipeline models, firstly recognizing drug/disease entity mentions and then identifying adverse drug events from drug/disease pairs. In this paper, we investigate joint models for simultaneously extracting drugs, diseases and adverse drug events. Compared with pipeline models, joint models have two main advantages. First, they make use of information integration to facilitate performance improvement; second, they reduce error propagation in pipeline methods. We compare a discrete model and a deep neural model for extracting drugs, diseases and adverse drug events jointly. Experimental results on a standard ADE corpus show that the discrete joint model outperforms a state-of-the-art baseline pipeline significantly. In addition, when discrete features are replaced by neural features, the recall is further improved.
BibTeX
@inproceedings{li2016joint,
  author = {Fei Li and Yue Zhang and Meishan Zhang and Donghong Ji},
  title = {{Joint Models for Extracting Adverse Drug Events from Biomedical Text}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2016},
  url = {https://openreview.net/forum?id=SkV1iQfdZB},
}
Figure