Pinpointing Fine-Grained Relationships between Hateful Tweets and Replies

Abdullah Albanyan, Eduardo BlancoAAAI 2022 · DOI

Introduction

The automated detection of hate speech has become a cornerstone of social media moderation. However, identifying a hateful post is only the first step in understanding the digital ecosystem. A critical, yet under-researched, area is the study of counter-hate—content intended to neutralize or challenge hateful narratives. Most existing research in this field relies on "synthetic" data, such as counter-narratives crafted by experts in controlled environments.

In reality, online discourse is far more volatile. Real-world interactions often involve sarcasm, personal vitriol, or the escalation of hate rather than calm, logical refutation. This research, conducted by teams at the University of North Texas and Arizona State University, addresses this gap by analyzing naturally occurring tweet-reply pairs to pinpoint the fine-grained relationships between hateful content and public responses.

The Dataset: Capturing Discourse in the Wild

To study these interactions, the authors curated a new, manually annotated corpus consisting of 5,652 (hateful tweet, reply) pairs. Unlike synthetic datasets like CONAN, this collection captures the informal and often aggressive nature of organic Twitter arguments.

Data was collected using two primary strategies: identifying replies to known hateful tweets from existing datasets and using the Twitter search API to find similar content. Each pair was then labeled for four binary attributes:

  1. Counter Hate?: Does the reply disagree with or challenge the original hate?
  2. Justification?: Does the reply provide a reason or counter-argument?
  3. Attacks Author?: Does the reply contain personal insults directed at the original poster?
  4. Additional Hate?: Does the reply introduce new hateful content or reinforce the original sentiment?
An example of a hateful tweet and subsequent replies from the dataset.

Methodology

The researchers employed a multi-pronged approach to classify these relationships:

  • Linguistic Analysis: Statistical t-tests were used to identify features—such as the use of the pronoun "you," profanity, and emoji polarity—that correlate with specific reply types.
  • Transformer-Based Classification: The study utilized BERT and BERTweet (a version of BERT pretrained on 850 million tweets). Models were tested using three input configurations: the hateful tweet alone, the reply alone, and the concatenated pair.
  • Multi-Task Pretraining: To enhance performance, the models underwent complementary task pretraining on datasets related to emotion detection, sentiment analysis, irony, and stance detection.

Key Insights and Results

The analysis revealed that constructive counter-speech is remarkably rare in the wild. As shown in the table below, only 20% of replies in the dataset actually countered the hateful content. Furthermore, when users did engage in counter-hate, they rarely provided a justification (21%) and frequently resorted to attacking the author (38%).

Distribution of binary labels across the 5,652 tweet-reply pairs.

A major finding of the study is that "Context is King." The model's ability to predict whether a reply would attack the author improved by 36.2% when it was provided with the original hateful tweet as context, rather than just the reply alone. This suggests that certain types of hate act as "bait" for specific aggressive responses.

The best-performing model was BERTweet when provided with both the tweet and the reply as input. It achieved high F1-scores across all tasks: 0.83 for Counter Hate, 0.88 for Justification, 0.80 for Attacks Author, and 0.91 for Additional Hate. Pretraining on "emotion" data was particularly effective for identifying counter-hate, while "stance" data helped the model recognize justifications.

Challenges and Future Directions

Despite the high performance of transformer models, several hurdles remain. Sarcasm and irony continue to be the primary causes of classification errors, as models often struggle to distinguish between genuine agreement and ironic dissent. Additionally, a significant portion of errors (up to 13%) stemmed from the models' inability to "see" multimodal content, such as images or videos referenced in the replies.

Future work will need to focus on multimodal models and the integration of broader world knowledge to better understand cultural and celebrity references. Ultimately, while this research categorizes what these replies are, the next vital step is measuring their effectiveness in curbing the spread of online hate.

Summary updated

BibTeX
@inproceedings{albanyan2022pinpointing,
  author = {Abdullah Albanyan and Eduardo Blanco},
  title = {{Pinpointing Fine-Grained Relationships between Hateful Tweets and Replies}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.1609/aaai.v36i10.21284},
}
Figure