A Social Media Study on the Effects of Psychiatric Medication Use

Koustuv Saha, Benjamin Sugar, John Torous, Bruno D. Abrahao, Emre Kiciman, Munmun De ChoudhuryICWSM 2019 · OpenReview

Abstract

Understanding the effects of psychiatric medications during mental health treatment constitutes an active area of inquiry. While clinical trials help evaluate the effects of these medications, many trials suffer from a lack of generalizability to broader populations. We leverage social media data to examine psychopathological effects subject to self-reported usage of psychiatric medication. Using a list of common approved and regulated psychiatric drugs and a Twitter dataset of 300M posts from 30K individuals, we develop machine learning models to first assess effects relating to mood, cognition, depression, anxiety, psychosis, and suicidal ideation. Then, based on a stratified propensity score based causal analysis, we observe that use of specific drugs are associated with characteristic changes in an individual’s psychopathology. We situate these observations in the psychiatry literature, with a deeper analysis of pre-treatment cues that predict treatment outcomes. Our work bears potential to inspire novel clinical investigations and to build tools for digital therapeutics.

Introduction

Psychiatry currently relies heavily on a "trial-and-error" approach to medication. While Randomized Controlled Trials (RCTs) are the gold standard for drug approval, they have significant blind spots: they often exclude 75% of real-world patients due to strict eligibility criteria, suffer from small sample sizes, and are too short to capture long-term behavioral shifts. This study addresses this "generalizability gap" by using social media as a passive sensor to understand how psychiatric drugs affect people in their daily lives, using digital footprints to measure symptomatic changes that clinical reports might miss.

Methodology

The researchers developed a multi-stage pipeline to transform unstructured Twitter data into a rigorous observational study.

The study pipeline, from drug identification to causal inference.
  1. Drug Identification and Intake Classification: A list of 49 FDA-approved antidepressants (SSRIs, SNRIs, TCAs, and TeCAs) was curated. A linear SVM classifier identified "first-person reports" of medication intake from millions of tweets.
  2. Symptom Modeling: Mental health signals were measured using lexical analysis (LIWC) for affect and cognition, and transfer learning. Classifiers trained on Reddit communities (e.g., r/depression, r/SuicideWatch) were applied to Twitter to detect anxiety, psychosis, and suicidal ideation.
  3. Causal Inference: To isolate drug effects, the study employed Stratified Propensity Score Analysis. "Treated" users were matched with "control" users based on pre-treatment behavior, including posting frequency, social network size, and baseline mental health scores.

Data Description

The study utilized a massive Twitter dataset (2015–2016) comprising approximately 300 million posts from 30,000 individuals who self-reported medication use. A control group of 300,000 users (707 million posts) was used for matching. Training data included 7,154 expert-annotated tweets for intake classification and tens of thousands of Reddit posts for symptom detection.

Results and Key Insights

The mental health classifiers demonstrated high performance, with F1-scores between 0.81 and 0.91. Propensity score matching successfully balanced the treatment and control groups, as evidenced by the reduction in standardized differences across covariates.

Classifier performance (ROC) and propensity score matching diagnostics.

Key findings include:

  • Relative Treatment Effect (RTE): For many users, starting medication was associated with an increase in negative affect and a decrease in cognitive clarity on social media.
  • SSRIs vs. TCAs: In this naturalistic setting, SSRIs were frequently associated with worsening symptoms or increased negative affect, while older TCAs showed more signs of improvement.
  • Predictive Cues: Pre-treatment language, such as high use of first-person singular pronouns ("I", "me"), predicted poorer outcomes for certain drugs like Sertraline (Zoloft).
  • Scale of Impact: Within the SSRI group, over 90% of users were in strata showing increased signals of anxiety and suicidal ideation post-treatment, a finding that provides large-scale evidence for clinical "black box" warnings.

Conclusion and Future Work

While this observational study cannot replace clinical evaluation due to selection bias and lack of dosage context, it highlights the potential of social media as a "digital phenotype." Future applications include "digital therapeutics" where clinicians monitor patient responses in real-time via opt-in social media tracking to catch adverse reactions much earlier than traditional scheduled visits.

Summary updated

Abstract

Understanding the effects of psychiatric medications during mental health treatment constitutes an active area of inquiry. While clinical trials help evaluate the effects of these medications, many trials suffer from a lack of generalizability to broader populations. We leverage social media data to examine psychopathological effects subject to self-reported usage of psychiatric medication. Using a list of common approved and regulated psychiatric drugs and a Twitter dataset of 300M posts from 30K individuals, we develop machine learning models to first assess effects relating to mood, cognition, depression, anxiety, psychosis, and suicidal ideation. Then, based on a stratified propensity score based causal analysis, we observe that use of specific drugs are associated with characteristic changes in an individual’s psychopathology. We situate these observations in the psychiatry literature, with a deeper analysis of pre-treatment cues that predict treatment outcomes. Our work bears potential to inspire novel clinical investigations and to build tools for digital therapeutics.
BibTeX
@inproceedings{saha2019social,
  author = {Koustuv Saha and Benjamin Sugar and John Torous and Bruno D. Abrahao and Emre Kiciman and Munmun De Choudhury},
  title = {{A Social Media Study on the Effects of Psychiatric Medication Use}},
  booktitle = {ICWSM},
  year = {2019},
  url = {https://openreview.net/forum?id=Y2j0A8cs7-},
}
Figure