Toxicity and networks of COVID-19 discourse communities: A tale of two media platforms

Karen DiCicco, Nahiyan Bin Noor, Niloofar Yousefi, Billy Spann, Nitin AgarwalROMCIR@ECIR 2023 · OpenReview

TL;DR

This study compares the toxicity of COVID-19 discourse on Twitter and Parler, finding Twitter to have higher overall toxicity despite its stricter moderation. The research highlights how conspiracy theories, particularly around vaccines, spread through distinct network structures on each platform, with Parler showing more clustered communities and "bridge nodes" facilitating misinformation.

What Problem Does This Paper Address?

The rise of toxic content and misinformation on social media platforms, especially concerning public health issues like COVID-19, poses a significant challenge. While platforms like Twitter have content moderation policies, others like Parler position themselves as "free speech" platforms with less stringent rules. This creates a need to understand how toxic discourse and misinformation spread differently across platforms with varying moderation approaches, and how these differences impact public health narratives. The paper aims to compare and analyze the toxicity and network structures of COVID-19 discourse on Twitter and Parler to shed light on these dynamics.

Methods

The researchers employed a multi-pronged approach:

  • Toxicity Analysis: They used the Detoxify model, a convolutional neural network trained on word vectors, to compute toxicity scores for each post/tweet. Detoxify provides scores for seven categories: toxicity (overall), severe toxicity, obscene, threat, insult, identity attack, and sexually explicit. Posts with a toxicity score of 0.5 or greater were considered toxic. They also explored Google's Perspective API for comparison.
  • Social Network Analysis: To understand the context and spread of toxic content, they constructed co-hashtag graph networks using the NetworkX Python library. The Girvan-Newman algorithm was used to identify distinct communities within these networks, focusing on posts with toxicity scores above 0.5.

Datasets

The study utilized data collected from January 1, 2020, to December 31, 2020, focusing on four categories of seed hashtags related to COVID-19: #f*ckcovid, #f*cklockdown, #f*ckmask, and #f*ckvaccine.

  • Twitter Data: Collected post-hoc using the Twitter Developer API.
  • Parler Data: Utilized an existing open dataset from Aliapoulios et al. (August 2018 - January 2021).

Key Insights and Ideas

  • Twitter's Unexpected Toxicity: Despite stricter moderation, Twitter exhibited higher overall toxicity in COVID-19 discourse compared to Parler, which was unexpected given Parler's reputation for leniency.
  • Platform-Specific Conspiracy Spread: While both platforms hosted toxic content, the nature of the spread differed. Twitter showed prominent conspiracy themes (Bill Gates, QAnon) emerging from vaccine discussions, while Parler displayed more clustered user communities and a distinct "misinformation echo chamber."
  • Bridge Nodes Facilitate Misinformation: Parler's network structure featured "bridge nodes" that acted as crucial connectors, spreading COVID-19 vaccine misinformation across different communities. A notable example was a bridge node connecting QAnon followers with a pro-Trump group.
  • Impact on Public Health Narratives: The study suggests that the way toxic content and misinformation are structured and spread on these platforms can influence public perception of non-pharmaceutical interventions (NPIs) and vaccination efforts.

Key Results

  • Higher Toxicity on Twitter: Twitter had a statistically significant higher level of toxicity across most COVID-19 discourse categories compared to Parler, except for the COVID dataset itself where Parler showed a higher percentage of toxic content.
  • Obscenity vs. Insult: Toxic content on both platforms was often obscene and insulting. Twitter's content was more frequently obscene, particularly in the "lockdown" category, while Parler's toxic content leaned more towards insults, especially in the "COVID" category.
  • Vaccine Discourse Differences: The percentage of toxic content within the vaccine discourse varied significantly, with Twitter showing 30.98% toxic content versus Parler's 11.93%.
  • Network Structure Contrasts: Twitter's co-hashtag network appeared unstructured and scattered, while Parler's was more clustered, featuring clear connective bridges and communities.

Limitations and Future Work

  • Seed Hashtag Constraint: The reliance on specific seed hashtags might limit the scope of the analysis, potentially missing other relevant toxic discussions.
  • Profanity vs. Intent: The Detoxify model sometimes struggles to distinguish between profanity used with intent and profanity that contributes to toxicity, potentially mislabeling content.
  • Future Directions: The authors propose expanding the analysis to include mention, URL, and retweet/echo networks to better identify misinformation spreaders. They also plan to investigate users suspected of being bots (using Botometer) and further explore vaccine and lockdown topics due to their high toxicity. Additionally, they aim to incorporate topic modeling and diffusion models to understand information spread on OSNs.

Summary updated

BibTeX
@inproceedings{dicicco2023toxicity,
  author = {Karen DiCicco and Nahiyan Bin Noor and Niloofar Yousefi and Billy Spann and Nitin Agarwal},
  title = {{Toxicity and networks of COVID-19 discourse communities: A tale of two media platforms}},
  booktitle = {ROMCIR@ECIR},
  year = {2023},
  url = {https://openreview.net/forum?id=YhjwuvsUye},
}
Figure