Echoes of Discord: Forecasting Hater Reactions to Counterspeech

Xiaoying Song, Sharon Lisseth Perez, Xinchen Yu, Eduardo Blanco, Lingzi HongNAACL 2025 · DOI

Introduction

Hate speech (HS) remains a pervasive issue on digital platforms. While "counterspeech"—direct, non-censorship-based responses—is frequently advocated as a primary mitigation strategy, its actual effectiveness in changing the behavior of the perpetrator is poorly understood. Most existing research focuses on bystander perception or the automated generation of counterspeech. This paper addresses a critical gap: predicting the "hater reentry" behavior. Specifically, it investigates whether a response intended to mitigate hate causes the perpetrator to disengage or, conversely, triggers a "backfire effect" where they return with even more vitriol.

The ReEco Dataset

To study these dynamics, the researchers curated the Reddit Echoes of Hate (ReEco) dataset. This dataset consists of 5,723 triple-turn conversation pairs extracted from 42 subreddits known for high HS prevalence. The structure of these interactions follows a specific sequence: an initial instance of hate speech, followed by a counterspeech response, and finally the hater's reaction (or lack thereof).

Figure 1: The triple-turn conversation structure used in the ReEco dataset, tracking the transition from initial hate speech to counterspeech and subsequent hater reentry.

The data spans five distinct community types: Discussion, Identity, Media-sharing, Meme, and Hobby. Analysis of the outcomes shows significant variance across these groups. For instance, the "Discussion" community saw the highest rate of hater disengagement (39% "No Reentry"), while the "Meme" community experienced the highest proportion of hateful reentries (27%).

Methodology

The researchers evaluated two primary modeling strategies to forecast hater reactions:

  1. Two-stage Reaction Predictor: A pipeline that first predicts if a hater will reenter (Binary: Yes/No) and then classifies the reentry as hateful or non-hateful.
  2. 3-way Classifier: A single model predicting one of three outcomes: No Reentry, Hateful Reentry, or Non-hateful Reentry.

Several architectures were tested, including standard BERT-base, BERT-MTL (Multi-Task Learning trained on toxicity and personal attack detection), and Llama 3 (8B) in both zero-shot and fine-tuned (LoRA) configurations. To uncover the linguistic drivers of these reactions, the team used SEANCE and spaCy to analyze sentiment and entity recognition.

Results and Analysis

The experimental results revealed a surprising performance gap between specialized smaller models and general-purpose Large Language Models (LLMs). The BERT-MTL model, particularly when using the "Pair" input (concatenating the original HS and the counterspeech), consistently outperformed Llama 3.

Figure 5: Performance comparison of different models on the binary reentry prediction task. BERT-MTL achieves the highest weighted average F1-score of 0.83.

While BERT-MTL achieved a weighted F1-score of 0.83 for predicting reentry, Llama 3 struggled significantly. The zero-shot Llama 3 performance was near-random, and even after fine-tuning, it only reached an F1 of 0.67 for the reentry task. This suggests that for nuance-heavy social classification tasks, models trained on domain-specific auxiliary tasks like toxicity detection still hold a significant edge.

Linguistic Drivers and the Backfire Effect

The study identified several key factors that influence hater behavior:

  • The Backfire Effect: Counterspeech that employs "enlightenment" language or fear-based appeals is statistically more likely to trigger a hater to re-engage.
  • Emotional Contagion: Haters often mirror the tone of the response. Aggressive counterspeech typically leads to hateful reentry, whereas responses incorporating respect, power, or forgiveness are more likely to elicit non-hateful reactions.
  • Rhetorical Challenges: Models frequently struggle with rhetorical questions. Because these are emotional expressions rather than information-seeking queries, models often misinterpret them, leading to false negatives in predicting disengagement.

Conclusion and Future Work

This research provides a vital roadmap for platform moderators and developers of automated counterspeech tools. It demonstrates that the way a response is framed is just as important as the intent to counter hate. However, the study is limited by its short-term "triple-turn" focus and does not account for long-term behavioral changes or the influence of broader thread context and user history. Future work will aim to categorize more nuanced behaviors, such as apologetic versus sarcastic reentries, to better understand the long-term "echoes" of digital discord.

Summary updated

BibTeX
@inproceedings{song2025echoes,
  author = {Xiaoying Song and Sharon Lisseth Perez and Xinchen Yu and Eduardo Blanco and Lingzi Hong},
  title = {{Echoes of Discord: Forecasting Hater Reactions to Counterspeech}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
  doi = {10.18653/v1/2025.findings-naacl.275},
}
Figure