Multi-Task Pharmacovigilance Mining from Social Media Posts

Shaika Chowdhury, Chenwei Zhang, Philip S. Yu

Abstract

Social media has grown to be a crucial information source for pharmacovigilance studies where an increasing number of people post adverse reactions to medical drugs that are previously unreported. Aiming to effectively monitor various aspects of Adverse Drug Reactions (ADRs) from diversely expressed social medical posts, we propose a multi-task neural network framework that learns several tasks associated with ADR monitoring with different levels of supervisions collectively. Besides being able to correctly classify ADR posts and accurately extract ADR mentions from online posts, the proposed framework is also able to further understand reasons for which the drug is being taken, known as 'indication', from the given social media post. A coverage-based attention mechanism is adopted in our framework to help the model properly identify 'phrasal' ADRs and Indications that are attentive to multiple words in a post. Our framework is applicable in situations where limited parallel data for different pharmacovigilance tasks are available. We evaluate the proposed framework on real-world Twitter datasets, where the proposed model outperforms the state-of-the-art alternatives of each individual task consistently.

Introduction

Pharmacovigilance—the science of monitoring drug safety—traditionally relies on clinical trials and voluntary reporting by healthcare professionals. However, these methods are often slow and suffer from significant under-reporting. Social media platforms like Twitter offer a real-time, "in-the-wild" look at patient experiences, but mining this data presents unique challenges. Patients rarely use formal medical terminology; instead of "insomnia," they might describe being "sleep deprived" or "never sleeping." These "phrasal" Adverse Drug Reactions (ADRs) are difficult for standard models to capture. Furthermore, ambiguity arises when a medical condition mentioned in a tweet could be either the reason for taking the drug (an Indication) or a side effect (an ADR). For instance, in the sentence "I took Paroxetine for my panic attacks but it made me fat," panic attacks is the indication, while fat is the ADR.

Methodology

The proposed framework utilizes a Multi-Task RNN Encoder-Decoder architecture to jointly learn three related tasks: ADR Classification (binary), ADR Labeling (sequence labeling), and Indication Labeling (sequence labeling).

The proposed multi-task neural network framework featuring a shared Bi-LSTM encoder and task-specific decoders with coverage-based attention.

Shared Representation and Embeddings

A Bidirectional LSTM (Bi-LSTM) serves as a shared encoder, processing tweets to create a semantic representation beneficial for all three tasks. To handle the informal nature of social media text, including slang and typos, the model combines standard word embeddings (GloVe) with character-level embeddings.

Coverage-based Attention

A key innovation is the coverage-based attention mechanism. Standard attention mechanisms often focus on single keywords. The coverage mechanism tracks which words the model has already attended to within a sliding window, forcing the model to recognize long, descriptive phrases rather than isolated words. This is particularly effective for capturing colloquial descriptions of health issues.

Experimental Setup

The study utilized the PSB 2016 Social Media Shared Task Twitter dataset. The ADR Classification task used a balanced set of 2,162 tweets. For ADR and Indication Labeling, a subset of 1,081 tweets was used, supplemented with 203 additional tweets from a 2015 study. The authors manually labeled the supplemental data to ensure consistency across tasks.

Results and Discussion

The multi-task framework consistently outperformed state-of-the-art baselines and single-task versions. As shown in the performance comparison, the multi-task model with attention and coverage (MT-Atten-Cov) achieved the highest performance across precision, recall, and F-1 metrics.

Performance comparison of different model variants on the ADR labeling task, showing the superiority of the multi-task attention-coverage model.

The most significant insight is that learning to identify Indications improves the model's ability to identify ADRs. By understanding the context of why a drug is taken, the model is less likely to mislabel an indication as a side effect. Multi-tasking also acts as a regularizer, preventing over-fitting and improving generalization. Qualitatively, the model successfully identified complex ADRs like "zombie" and "heart was hurting" in contexts where baseline models failed.

Conclusion

While the model shows strong performance in ADR classification and labeling, Indication Labeling scores were lower, likely due to data sparsity as many users do not mention their reasons for taking a drug. Future work could involve incorporating additional tasks, such as identifying specific drug-ADR relationships or drug-drug interactions, to further refine the model's capabilities in pharmacovigilance.

Summary updated

Abstract

Social media has grown to be a crucial information source for pharmacovigilance studies where an increasing number of people post adverse reactions to medical drugs that are previously unreported. Aiming to effectively monitor various aspects of Adverse Drug Reactions (ADRs) from diversely expressed social medical posts, we propose a multi-task neural network framework that learns several tasks associated with ADR monitoring with different levels of supervisions collectively. Besides being able to correctly classify ADR posts and accurately extract ADR mentions from online posts, the proposed framework is also able to further understand reasons for which the drug is being taken, known as 'indication', from the given social media post. A coverage-based attention mechanism is adopted in our framework to help the model properly identify 'phrasal' ADRs and Indications that are attentive to multiple words in a post. Our framework is applicable in situations where limited parallel data for different pharmacovigilance tasks are available.We evaluate the proposed framework on real-world Twitter datasets, where the proposed model outperforms the state-of-the-art alternatives of each individual task consistently.
BibTeX
@inproceedings{chowdhury2018multi,
  author = {Shaika Chowdhury and Chenwei Zhang and Philip S. Yu},
  title = {{Multi-Task Pharmacovigilance Mining from Social Media Posts}},
  booktitle = {The Web Conference},
  year = {2018},
  url = {https://openreview.net/forum?id=HJNMUzZuZS},
}
Figure