Deep Learning for Social Media Health Text Classification

Santosh T.y.s.s, Santosh Tokala, Vaibhav Gambhir, Animesh MukherjeeEMNLP 2018

TL;DR

This paper explores deep learning models, specifically CNN-LSTM and attention-based LSTMs, for classifying social media posts related to medication. They achieved strong results on two tasks: identifying posts mentioning drugs and categorizing medication intake (personal, possible, or none), demonstrating the power of deep learning for nuanced health-related text analysis.

What Problem Does This Paper Address?

The paper tackles the challenge of extracting meaningful health information from the noisy and informal text found on social media. Specifically, it focuses on "pharmaco-vigilance," the process of monitoring and detecting adverse drug reactions or medication-related information. Social media platforms are a rich source of such data, but the language used is often informal, contains misspellings, and exhibits complex linguistic variations, making traditional NLP methods struggle. The authors aim to leverage deep learning's ability to learn patterns from unlabelled data to overcome these challenges in classifying medication-related posts.

Methods

The authors propose and evaluate two main deep learning architectures:

  • CNN-LSTM: This model combines Convolutional Neural Networks (CNNs) with Long Short-Term Memory (LSTM) networks. The CNNs are used to extract higher-level phrase representations from word embeddings, which are then fed into the LSTM to capture sequential dependencies and form a sentence-level representation. This approach aims to leverage the strengths of both architectures: CNNs for local feature extraction and LSTMs for understanding long-term context.
  • LSTM with Attention Mechanism: This architecture builds upon the standard LSTM by incorporating an attention mechanism. Traditional LSTMs compress the entire input sequence into a fixed-length representation, which can be a bottleneck for long sequences. The attention mechanism allows the model to selectively focus on different parts of the input sequence at each step of generating the output, enabling a more direct connection between input and output and improving the handling of long-term dependencies.

Both architectures were experimented with at both the word and character levels. They also explored various pre-trained word embeddings (GloVe, Word2Vec) and task-specific embeddings learned by the models themselves.

Datasets

The paper uses datasets from the 3rd Social Media Mining for Health Applications (SMM4H) Shared Task at EMNLP 2018. The datasets are divided into two tasks:

  • Task 1: A binary classification task to detect posts mentioning any drug name or dietary supplement.
  • Task 2: A multi-class classification task to categorize tweets into three classes: personal medication intake, possible medication intake, and non-intake.

The paper provides statistics for the training and validation sets for both tasks, detailing the number of posts in each category.

Key Insights and Ideas

  • Deep Learning for Nuanced Health Text: The paper strongly suggests that deep learning models are well-suited for understanding the complexities of health-related discussions on social media, going beyond simple keyword matching.
  • Hybrid Architectures Excel: Combining CNNs and LSTMs (CNN-LSTM) proved effective, demonstrating the benefit of leveraging both local feature extraction and sequential modeling.
  • Attention Enhances Contextual Understanding: The attention mechanism in LSTMs significantly improved the model's ability to handle long-term dependencies and contextual nuances in medication-related tweets.
  • Character-Level Embeddings Offer Robustness: Experimenting with character-level embeddings showed promise, potentially making the models more robust to misspellings and variations common in social media text.

Key Results

The paper reports strong performance on both tasks. While specific top-ranking results are presented in tables, some highlights include:

  • For Task 1, the character-level LSTM-CNN achieved good precision and F1-score on the test data.
  • For Task 2, the character-level LSTM-CNN achieved the highest micro-averaged precision, recall, and F1-score on the test data, indicating its superior performance in classifying medication intake.
  • The paper also shows that different embedding strategies (GloVe, Word2Vec, task-specific) and classifiers (GBDT, Naive Bayes) yield varying results, highlighting the importance of hyperparameter tuning and model selection.

Limitations and Future Work

  • Limitations: The paper acknowledges that while deep learning has advanced the field, further optimization and performance improvements are still needed. The inherent noisiness and informality of social media data remain a persistent challenge.
  • Future Work: The authors suggest exploring how the advantages of deep learning can be further expanded to address other pertinent challenges in social media text analysis. This implies a continued focus on developing more sophisticated models that can handle even more complex linguistic phenomena and a wider range of health-related topics.

Summary updated

BibTeX
@inproceedings{tyss2018deep,
  author = {Santosh T.Y.S.S and Santosh Tokala and Vaibhav Gambhir and Animesh Mukherjee},
  title = {{Deep Learning for Social Media Health Text Classification}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
}
Figure