When the Echo Chamber Shatters: Examining the Use of Community-Specific Language Post-Subreddit Ban

Milo Trujillo, Sam Rosenblatt, Guillermo de Anda Jáuregui, Emily Moog, Briane Paul V. Samson, Laurent Hébert-Dufresne, Allison M. RothACL 2021

TL;DR

This paper introduces a novel, unsupervised method to identify "in-group" language within online communities and uses it to analyze how users' language and activity change after a subreddit ban. They found that while top users significantly reduce their activity, the impact on language use is more varied, suggesting that bans don't affect all users or communities equally.

What Problem Does This Paper Address?

Online platforms like Reddit host countless communities, some of which can become breeding grounds for harmful speech and harassment. Platform administrators and moderators often resort to banning entire subreddits to curb such behavior. However, the effectiveness of these bans is not well understood, with mixed results from previous studies. A key challenge is how to objectively measure the impact of a ban, especially on language use, without relying on subjective definitions of "harmful speech." This paper aims to provide a more flexible and automated way to assess the linguistic and behavioral shifts in communities after a ban, moving beyond simple activity metrics.

Methods

The authors developed a flexible, unsupervised methodology to identify a community's unique linguistic signature, termed "in-group language." This involves comparing word frequencies within a specific subreddit to a large baseline corpus of general Reddit activity. They use Jensen-Shannon Divergence (JSD) to identify words that are disproportionately overrepresented in a given subreddit, effectively capturing its unique vocabulary. This "in-group vocabulary" is not necessarily indicative of harmful speech but serves as a linguistic fingerprint.

They then applied this method to analyze user behavior before and after the ban of 15 subreddits. For each subreddit, they tracked the activity (number of comments) and in-group language usage of two groups: the top 100 most active users and 1000 randomly sampled users. They calculated changes in activity and in-group language use on a scale from -1 to 1, where -1 indicates a complete shift to pre-ban or post-ban behavior, respectively. Statistical models (generalized linear mixed models and likelihood ratio tests) were used to determine the significance of these changes and the influence of user engagement (top vs. random) and subreddit identity.

Datasets

The study utilized data from 15 subreddits that were banned in June 2020 as part of Reddit's "Great Ban." For the baseline corpus, they used 70 million non-bot comments from across all of Reddit during the same time frame. The Pushshift API was used to collect submissions and comments from the banned subreddits for the 182 days preceding their ban.

Key Insights and Ideas

  • "In-group language" as a linguistic fingerprint: The concept of using unique vocabulary as a community's signature is a powerful way to track its identity and evolution without needing to pre-define "harmful" content.
  • Heterogeneity of ban impact: Bans are not a one-size-fits-all solution. The study highlights significant differences in how various subreddits and different types of users within those subreddits respond to bans.
  • Top users are more affected: The most active users ("top users") consistently showed greater reductions in both posting activity and, to a lesser extent, in-group language use compared to random users. This suggests that while bans might silence the most vocal members, they don't necessarily eliminate the community's core language.
  • Content matters: The type of content a subreddit hosted played a role in how its users reacted. Subreddits focused on white supremacy and fascism were more significantly impacted than those focused on "dark jokes" or "anti-political" content.

Key Results

  • Activity Reduction: Top users consistently showed a greater decrease in posting activity post-ban compared to random users across all studied subreddits.
  • Language Shift: While many subreddits saw a decrease in in-group language usage, this effect was less consistent than the activity reduction, particularly for random users.
  • Subreddit-Specific Effects:
    • "Dark joke" subreddits were largely unaffected by bans.
  • "Anti-political" subreddits showed moderate impacts, with top users becoming less active and random users using less in-group language.
  • "Extreme right-wing" communities (e.g., white supremacy, fascism) experienced the most significant impact, with substantial drops in both activity and in-group language use.
  • Statistical Significance: Likelihood ratio tests confirmed significant differences in both activity and vocabulary shifts between top and random users for most subreddits.

Limitations and Future Work

  • Sample Size: The study analyzed 15 subreddits, which, while larger than many previous studies, is still a limited sample. The authors suggest that including subreddits with fewer users would make quantitative comparisons less consistent, but a larger sample could reveal more robust trends.
  • Defining "In-group Language": While JSD is a robust method, the authors acknowledge that it might not capture all nuances of in-group language, and further computational linguistics research could refine these techniques.
  • User Demographics and Network Position: Future work could explore how user demographics (age of account, inferred gender, etc.) and their position within community interaction networks influence their response to bans.
  • Longer-Term Impacts: The study focused on a 60-day window post-ban. Investigating longer-term effects, including whether communities attempt to "rebuild" elsewhere or integrate into existing platforms, would be valuable.
  • Qualitative Approaches: The authors suggest that embracing more holistic, qualitative methodologies (from anthropology, sociology, etc.) could provide deeper insights into community and user responses to censorship.

Summary updated

BibTeX
@inproceedings{trujillo2021when,
  author = {Milo Trujillo and Sam Rosenblatt and Guillermo de Anda Jáuregui and Emily Moog and Briane Paul V. Samson and Laurent Hébert-Dufresne and Allison M. Roth},
  title = {{When the Echo Chamber Shatters: Examining the Use of Community-Specific Language Post-Subreddit Ban}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
}
Figure