An NLP-assisted Bayesian time-series analysis for prevalence of Twitter cyberbullying during the COVID-19 pandemic

Christopher Perez, Sayar KarmakarSoc. Netw. Anal. min 2023 · DOI

TL;DR

This paper uses NLP and Bayesian time series analysis to track Twitter cyberbullying trends from 2019-2021, revealing seasonal patterns that were amplified during the COVID-19 pandemic. By combining keyword searches with a hatefulness/offensiveness NLP model, the study provides a more nuanced understanding of cyberbullying prevalence beyond simple keyword counts.

What Problem Does This Paper Address?

The authors tackle the challenge of accurately measuring and understanding the prevalence of cyberbullying, particularly in the context of the COVID-19 pandemic. Traditional methods like surveys can suffer from biases, and simply counting tweets with cyberbullying-related keywords might not capture the true extent or nature of the problem. The rapid shift to online interactions during the pandemic also raised questions about its impact on cyberbullying. This paper aims to provide a more robust, data-driven approach to analyze these trends over a significant period.

Methods

The core of the methodology involves a multi-step process:

  • Data Collection: Tweets were collected using the Twitter API's search endpoint, focusing on keywords associated with cyberbullying. To address potential sampling biases of the search endpoint, the count endpoint was also used to get a more reliable daily total of relevant tweets.
  • NLP Pre-processing: A pre-trained NLP model (specifically, models available on HuggingFace) was employed to classify tweets based on their "hatefulness" and "offensiveness" scores. This step aims to filter out irrelevant tweets and identify those more likely to represent actual cyberbullying.
  • Time Series Construction: The proportion of tweets identified as hateful or offensive (above certain thresholds) was calculated daily. This proportion was then multiplied by the daily tweet counts from the count endpoint to create a more accurate time series representing the prevalence of potentially abusive content.
  • Bayesian Time Series Modeling: A time-varying Bayesian autoregressive count (TVBARC) model was used to analyze the constructed time series. This model is suitable for count data, accounts for temporal dependencies (lags), and can capture subtle changes over time.

Datasets

  • Twitter Data: 1,004,466 tweets were collected from January 1st, 2019, to December 31st, 2021, using the Twitter API.
  • COVID-19 Case Data: Daily confirmed COVID-19 cases in the US were sourced from Johns Hopkins University CSSE COVID-19 Data (via Ritchie et al. [41]) for comparative analysis.

Key Insights and Ideas

  • NLP as a Crucial Filter: Simply relying on keyword searches for cyberbullying is insufficient. Using NLP to assess hatefulness and offensiveness provides a much more refined measure, helping to distinguish between general discussion and actual abusive content.
  • Seasonal Patterns Amplified by COVID-19: The study found strong weekly and yearly seasonality in hateful speech on Twitter. While these patterns existed before the pandemic, they appear to have been amplified or altered in magnitude and timing during COVID-19.
  • Disruption vs. Continuation: The findings suggest that while cyberbullying discourse may have seen peaks coinciding with COVID-19 case surges, the overall trend might indicate a decrease in the proportion of hateful tweets over time, potentially aligning with findings that suggest the pandemic disrupted some forms of online aggression.
  • Lagged Effects are Key: The Bayesian time series models revealed significant lagged effects, indicating that past tweets influence current trends. Identifying these lags is crucial for understanding the dynamics of cyberbullying.

Key Results

  • The analysis revealed consistent weekly and yearly seasonality in hateful speech on Twitter across 2019-2021.
  • The COVID-19 pandemic appears to have influenced the timing and magnitude of these seasonal peaks, with a notable peak in April 2020.
  • While raw tweet counts showed fluctuations, the filtered data (using NLP scores) provided a clearer picture of potentially abusive content.
  • The study found that the proportion of hateful tweets, when adjusted for daily sample size, showed a slight downward trend leading up to and during the pandemic.
  • The models identified significant weekly effects (lags 5, 6, 7, 13, 14) when focusing on hateful tweets, which were less apparent in the raw data.

Limitations and Future Work

  • Defining Cyberbullying: The study assumes that "hatefulness" and "offensiveness" are good proxies for cyberbullying, but acknowledges that accurately detecting cyberbullying is complex and may require more advanced models that consider user profiles, comment history, and network structures.
  • NLP Model Bias: The NLP model used may exhibit racial biases present in its training data, potentially affecting its accuracy in identifying offensive or hateful content, especially concerning certain demographic groups.
  • Causality: The study can only infer correlations between COVID-19 cases and cyberbullying trends, not establish causal relationships.
  • Twitter Sampling: The unknown sampling algorithm of the Twitter API introduces uncertainty about the exact population the results generalize to.
  • Future Work: The authors suggest using more complex prediction models, integrating spatial analysis to study regional differences, and employing Vector Auto-Regression (VAR) models to simultaneously analyze multiple time series (e.g., COVID-19 cases, abusive tweets, search engine data). They also recommend exploring seasonal models to better capture the data's dynamics.

Summary updated

BibTeX
@article{perez2023nlp,
  author = {Christopher Perez and Sayar Karmakar},
  title = {{An NLP-assisted Bayesian time-series analysis for prevalence of Twitter cyberbullying during the COVID-19 pandemic}},
  journal = {Soc. Netw. Anal. min},
  year = {2023},
  doi = {10.1007/s13278-023-01053-4},
}
Figure