An Ensemble Deep Learning Model for Drug Abuse Detection in Sparse Twitter-Sphere

Han Hu, NhatHai Phan, James Geller, Stephen Iezzi, Huy T. Vo, Dejing Dou, Soon Ae Chun

Abstract

As the problem of drug abuse intensifies in the U.S., many studies that primarily utilize social media data, such as postings on Twitter, to study drug abuse-related activities use machine learning as a powerful tool for text classification and filtering. However, given the wide range of topics of Twitter users, tweets related to drug abuse are rare in most of the datasets. This imbalanced data remains a major issue in building effective tweet classifiers, and is especially obvious for studies that include abuse-related slang terms. In this study, we approach this problem by designing an ensemble deep learning model that leverages both word-level and character-level features to classify abuse-related tweets. Experiments are reported on a Twitter dataset, where we can configure the percentages of the two classes (abuse vs. non abuse) to simulate the data imbalance with different amplitudes. Results show that our ensemble deep learning models exhibit better performance than ensembles of traditional machine learning models, especially on heavily imbalanced datasets.

Introduction

Public health monitoring increasingly relies on social media to track drug abuse epidemics in real-time. However, identifying relevant content in the "Twitter-Sphere" is a "needle in a haystack" problem. Tweets related to actual drug abuse are extremely sparse compared to general discourse, leading to a significant class imbalance. Furthermore, the use of evolving slang, typos, and informal language poses a challenge for traditional text classifiers. This study addresses these issues by developing a robust ensemble deep learning model designed to maintain performance even in highly imbalanced data environments.

Methodology

The proposed model integrates multiple neural architectures and auxiliary features to capture the nuances of drug-related discourse.

  1. Word-level CNN (W-CNN): Processes tweets as word sequences using pre-trained "Drug Chatter" embeddings (trained on 1 billion tweets) to capture semantic context.
  2. Character-level CNN (C-CNN): Analyzes individual characters to provide resilience against creative misspellings and slang (e.g., "c0ke").
  3. Auxiliary Features: Hand-crafted features, including slang lexicons, synonym expansions via WordNet, and medical indicators, are injected into the final hidden layer to guide the network.
Figure 1: Architecture and parameters of the ensemble deep learning model, featuring W-CNN and C-CNN branches alongside auxiliary feature integration.

The final classification is determined through majority voting across independently trained model instances, enhancing stability and reducing overfitting.

Experimental Design and Datasets

The study utilized a benchmark dataset of 4,736 tweets (2,657 positive, 2,079 negative), curated through manual labeling and Amazon Mechanical Turk. To evaluate the model's robustness in real-world scenarios, the authors simulated varying levels of data sparsity by creating five dataset versions with different positive-to-negative ratios.

Table 1: Class distribution splits used to simulate varying levels of data imbalance for model evaluation.

Results and Discussion

The Ensemble CNN was compared against an ensemble of traditional machine learning models (SVM, Random Forest, Naive Bayes).

  • Balanced Performance: In a 50:50 split, traditional ML slightly outperformed the CNN (F1-score 0.8623 vs. 0.8520).
  • Imbalanced Robustness: As sparsity increased, the CNN model demonstrated superior stability. At a 10:90 split, the Ensemble CNN achieved an F1-score of 0.5389, significantly higher than the traditional ensemble's 0.4999.
  • Feature Importance: Character-level features were crucial for handling the informal nature of Twitter data. Additionally, domain-specific embeddings ("Drug Chatter") significantly outperformed general-purpose embeddings like Google News or GloVe, highlighting the importance of in-domain training data.

Conclusion

The ensemble deep learning approach provides a more reliable framework for public health monitoring in sparse data environments. While traditional models perform well on balanced data, the deep learning ensemble's resilience to class imbalance makes it better suited for real-world social media analysis. Future work will explore more complex ensemble strategies and the application of this model to large-scale, real-time Twitter data for geographic trend mapping.

Summary updated

Abstract

As the problem of drug abuse intensifies in the U.S., many studies that primarily utilize social media data, such as postings on Twitter, to study drug abuse-related activities use machine learning as a powerful tool for text classification and filtering. However, given the wide range of topics of Twitter users, tweets related to drug abuse are rare in most of the datasets. This imbalanced data remains a major issue in building effective tweet classifiers, and is especially obvious for studies that include abuse-related slang terms. In this study, we approach this problem by designing an ensemble deep learning model that leverages both word-level and character-level features to classify abuse-related tweets. Experiments are reported on a Twitter dataset, where we can configure the percentages of the two classes (abuse vs. non abuse) to simulate the data imbalance with different amplitudes. Results show that our ensemble deep learning models exhibit better performance than ensembles of traditional machine learning models, especially on heavily imbalanced datasets.
BibTeX
@inproceedings{hu2019ensemble,
  author = {Han Hu and NhatHai Phan and James Geller and Stephen Iezzi and Huy T. Vo and Dejing Dou and Soon Ae Chun},
  title = {{An Ensemble Deep Learning Model for Drug Abuse Detection in Sparse Twitter-Sphere}},
  booktitle = {MedInfo},
  year = {2019},
  url = {https://openreview.net/forum?id=razZBmWtkZ5},
}
Figure