Online conspiracy communities are more resilient to deplatforming

Corrado Monti, Matteo Cinelli, Carlo Michele Valensise, Walter Quattrociocchi, Michele Starnini

Introduction

Social media platforms frequently employ "deplatforming"—the permanent banning of users or entire communities—as a primary tool to curb the spread of harmful content. While existing research suggests that these bans effectively reduce toxicity within the platform that issues them, the long-term consequences for the displaced users remain a subject of intense debate. Do these communities simply dissipate, or do they migrate to less-moderated spaces and become more radicalized?

This study addresses a critical gap in our understanding of content moderation by examining how the nature of a community—specifically, whether it is built around a shared conspiracy theory or a shared hobby of harassment—influences its "resilience," or its ability to survive and rebuild after a major disruption.

Comparative Case Studies: QAnon vs. FatPeopleHate

To measure resilience, researchers tracked the migration of users from two massive subreddits banned by Reddit: r/GreatAwakening (a central hub for QAnon conspiracy theorists) and r/FatPeopleHate (FPH, a community dedicated to bullying). Both groups primarily migrated to Voat, a now-defunct Reddit clone known for its minimal moderation and "free speech" ethos.

The researchers utilized several technical approaches to track these migrations:

  • User Matching: By identifying users who maintained the exact same username on both Reddit and Voat, the authors tracked a subset of "zealots" to represent broader community behavior.
  • Behavioral Classification: Users were categorized based on whether they stayed on Reddit, migrated to Voat, or abandoned the mainstream platform entirely.
  • Toxicity Analysis: Using the IMSyPP classifier, the study measured changes in the frequency of inappropriate, offensive, or violent language before and after the ban.
  • Social Network Reconstruction: The authors built bipartite networks (linking users to the posts they interacted with) to determine if the social structures on Voat were deliberate recreations of the original Reddit networks or merely random associations.

Key Findings: Identity vs. Hobby

The most significant discovery was the stark difference in commitment between the two groups. For members of FatPeopleHate, participation appeared to be a "casual activity." In contrast, members of the QAnon community viewed their participation as a core component of their identity.

1. Migration and Abandonment Rates

Conspiracy theorists were far more likely to abandon the mainstream platform following the ban. Over 40% of QAnon users left Reddit entirely, compared to only 20% of FPH users. Furthermore, 20% of QAnon users successfully "fully migrated" (leaving Reddit to join Voat), while only 4% of FPH users did the same. This suggests that the conspiracy community was much more willing to sacrifice the convenience of a mainstream platform to maintain their social ties.

2. The Portability of Social Capital

The study found that conspiracy theorists did not just move to a new site; they actively sought out their former social circles. The QAnon community reconstructed 16.7% of their Reddit social network on Voat. Statistical analysis using null models yielded a z-score of 6.6, indicating that this reconstruction was a deliberate and highly resilient effort. Conversely, the FPH network reconstruction (9.4%) resulted in a z-score of 0.86, meaning their new social structure on Voat was essentially indistinguishable from random noise.

3. The Toxicity Surge

While deplatforming removed toxic content from Reddit, it led to a surge in toxicity on the alternative platform. The average fraction of toxic comments rose by 8 percentage points for QAnon and 10 percentage points for FPH after the move to Voat. This highlights the "polarization trap": by forcing these groups into unmoderated echo chambers, deplatforming may inadvertently fuel narratives of persecution and increase the radicalization of the remaining members.

The "Zealot" Growth Engine

The survival of the conspiracy community on Voat was not driven by the masses, but by a small group of highly active "zealots." The top 1% of users became significantly more active after the migration, serving as the engine that kept the community alive in its new environment. This suggests that the resilience of a community is often tied to the dedication of its most extreme members.

Conclusion

The study concludes that while deplatforming acts as an effective "firewall" for mainstream platforms—protecting the general user base from harmful content—it can also act as a "crucible" for conspiracy communities. For groups built on shared identity and perceived persecution, a ban may result in a smaller, more tightly knit, and significantly more toxic community. Future moderation strategies must account for these different levels of resilience to avoid inadvertently strengthening the very groups they seek to marginalize.

Summary updated

Abstract

Online social media foster the creation of active communities around shared narratives. Such communities may turn into incubators for conspiracy theories-some spreading violent messages that could sharpen the debate and potentially harm society. To face these phenomena, most social media platforms implemented moderation policies, ranging from posting warning labels up to deplatforming, i.e. permanently banning users. Assessing the effectiveness of content moderation is crucial for balancing societal safety while preserving the right to free speech. In this article, we compare the shift in behavior of users affected by the ban of two large communities on Reddit, <i>GreatAwakening</i> and <i>FatPeopleHate</i>, which were dedicated to spreading the QAnon conspiracy and body-shaming individuals, respectively. Following the ban, both communities partially migrated to Voat, an unmoderated Reddit clone. We estimate how many users migrate, finding that users in the conspiracy community are much more likely to leave Reddit altogether and join Voat. Then, we quantify the behavioral shift within Reddit and across Reddit and Voat by matching common users. While in general the activity of users is lower on the new platform, <i>GreatAwakening</i> users who decided to completely leave Reddit maintain a similar level of activity on Voat. Toxicity strongly increases on Voat in both communities. Finally, conspiracy users migrating from Reddit tend to recreate their previous social network on Voat. Our findings suggest that banning conspiracy communities hosting violent content should be carefully designed, as these communities may be more resilient to deplatforming.
BibTeX
@inproceedings{monti2023online,
  author = {Corrado Monti and Matteo Cinelli and Carlo Michele Valensise and Walter Quattrociocchi and Michele Starnini},
  title = {{Online conspiracy communities are more resilient to deplatforming}},
  booktitle = {PNAS},
  year = {2023},
  doi = {10.1093/pnasnexus/pgad324},
}
Figure