Using Matched Samples to Estimate the Effects of Exercise on Mental Health via Twitter

Virgile Landeiro Dos Reis, Aron CulottaAAAI 2015 · DOI

Abstract

Observational health studies on social media often suffer from selection bias. This paper presents a framework for conducting "Web-scale" studies by applying statistical matching techniques to Twitter data. By creating "virtual twins" for exercisers, we isolate the effect of physical activity on mood, finding that regular exercise correlates with significantly lower levels of anxiety and dejection.

Introduction

While social media is a powerful tool for health monitoring, its utility for causal inference is limited by the non-random nature of user behavior. Traditional clinical studies use randomized controlled trials (RCTs) to avoid the "correlation vs. causation" fallacy. On Twitter, users who post about exercise likely differ demographically and socially from those who do not. This study addresses these confounding variables by developing a matching pipeline to estimate the impact of exercise on mental health.

Methodology

The researchers identified a "treatment group" of 67,000 unique users via fitness app hashtags (e.g., #nikeplus, #strava). To mitigate confounding bias, each exerciser was paired with a "virtual twin" from a control group who shared similar characteristics but did not use fitness apps.

Matching Criteria

Users were matched based on:

  • Gender: Inferred from U.S. Census name data.
  • Location: Matched by city and state.
  • Social Activity: Matched using follower counts, friend counts, and total tweet volume.
Comparison of demographic and activity metrics across the exercise, matched, and random groups.

Mood Classification

A text classifier categorized tweets into three states: Hostility, Dejection, and Anxiety. Based on the Profile of Mood States (POMS), a Logistic Regression model was trained using unigrams, bigrams, and LIWC features, achieving an average accuracy of 87%.

Key Results

The study compared the mood of 1,161 physically active users against 1,161 matched controls.

  • Dejection: Exercisers showed a 3.9% decrease compared to the matched control ($p < 0.001$).
  • Anxiety: Exercisers showed a 2.7% decrease ($p = 0.02$).
  • Hostility: No significant difference was observed.

These findings align with psychological literature suggesting exercise alleviates mild depression and anxiety but has a less clear effect on anger.

The Importance of Matched Sampling

A critical insight of this research is the danger of naive sampling. When comparing exercisers to a purely random control group, the effects were overestimated by 2 to 20 times. As shown in the demographic analysis in the figure above, random users typically have significantly fewer friends and a much higher proportion of female users than the exercise-focused group. Matching on social activity (followers/friends) serves as a proxy for "online personality" or social status, which is essential for isolating the behavior of interest.

Conclusion and Future Work

This work demonstrates that with proper statistical controls, social media can provide valuable insights into public health. However, limitations remain, including selection bias (only app-using exercisers are tracked) and unobserved confounders like diet or socioeconomic status. Future research could utilize longitudinal data to track individual mood changes over time or employ Propensity Score Matching for even more refined control groups.

Summary updated

BibTeX
@inproceedings{reis2015using,
  author = {Virgile Landeiro Dos Reis and Aron Culotta},
  title = {{Using Matched Samples to Estimate the Effects of Exercise on Mental Health via Twitter}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2015},
  doi = {10.1609/aaai.v29i1.9205},
}
Figure