Uncovering the Root of Hate Speech: A Dataset for Identifying Hate Instigating Speech

Hyoungjun Park, Ho Shim, Kyuhan LeeEMNLP 2023 · DOI

Introduction

Traditional content moderation often operates reactively, focusing on identifying and removing individual instances of hate speech after they have already been posted. This "whack-a-mole" approach frequently overlooks the underlying catalysts: posts that may not be explicitly hateful themselves but are designed to provoke a cascade of toxicity in the comments section. Researchers have termed this phenomenon Hate Instigating Speech (HIS).

Identifying HIS is challenging because it often relies on nuance, sarcasm, or "trolling" tactics that bypass standard hate-speech filters. To address this gap, a new study introduces a multilingual dataset (English and Korean) specifically designed to map the relationship between "parent" comments and their "child" replies, allowing for the detection of these root causes of online toxicity.

Comparison between traditional hate speech detection and the proposed hate instigating speech detection.

Methodology and Annotation

The researchers framed the identification of HIS as a regression task, aiming to predict the intensity of instigation rather than a simple binary classification. The ground truth was established through a rigorous two-step human annotation process:

  1. Reply Labeling: Annotators categorized thousands of YouTube replies as either "Hate Speech" or "Non-Hate Speech."
  2. HIS Scoring: The "Hate Instigating" score for a parent comment was calculated as the ratio of hateful replies it received relative to its total number of replies.

To evaluate detection capabilities, the authors tested various baseline models, including traditional linear models (Lasso), tree-based methods (Random Forest, XGBoost), and advanced deep learning architectures like CNNs, LSTMs, and pre-trained BERT Regressors.

Dataset Overview

The dataset comprises 4,165 parent comments and 11,310 associated replies collected from YouTube news channels between January and April 2022. The sources were selected to represent a spectrum of political leanings: CNN and Fox News for English, and JTBC and TV Chosun for Korean. Unlike datasets that rely on automated "silver labels," this corpus utilizes manual human labeling of entire conversation threads to ensure high data quality.

The "Non-Hate" Instigator

A pivotal finding of the research is that a significant portion of comments triggering hate speech are not overtly hateful. In the English dataset, the majority of instigating comments were classified as "non-hate HIS." This confirms that moderation systems focusing solely on explicit hate speech are missing the primary drivers of toxic discourse.

Statistical breakdown of Hate HIS vs. Non-hate HIS across English and Korean datasets.

As shown in the data, for the English corpus, 721 out of 976 instigating comments did not contain hate speech themselves. This highlights the subtlety of instigation, which often employs sarcasm or satire to provoke others.

Key Insights and Results

The study revealed several critical aspects of HIS:

  • Linguistic Complexity: Detecting HIS in English proved significantly more difficult than in Korean. English instigators frequently used complex linguistic devices like sarcasm, whereas Korean instigators tended to be more direct.
  • Political Correlation: Right-leaning media channels in both the US (Fox News) and South Korea (TV Chosun) exhibited a higher proportion of hate-instigating comments and a higher ratio of hateful replies compared to their left-leaning counterparts.
  • Engagement Metrics: HIS comments consistently garnered more engagement (replies) than non-instigating comments, suggesting that provocative content is a major driver of platform interaction.

Performance was measured using Mean Absolute Error (MAE). Models struggled with the English dataset (MAE ~0.42–0.46), while performing better on the Korean data (MAE ~0.25). While the BERT Regressor was the top performer, it still faced difficulties capturing the contextual nuances required for accurate English HIS detection.

Limitations and Future Work

The current research focuses on the immediate relationship between a parent comment and its direct replies. Future studies could explore the "chain of instigation," where hateful replies trigger further toxicity. Additionally, expanding the scope to multimodal data (images and memes) and categorizing instigation into specific types (e.g., sexism or racism) would provide more granular insights. Finally, the authors note that moderating "non-hateful" but "instigating" speech presents significant ethical challenges regarding free speech that platforms must carefully navigate.

Summary updated

BibTeX
@inproceedings{park2023uncovering,
  author = {Hyoungjun Park and Ho Shim and Kyuhan Lee},
  title = {{Uncovering the Root of Hate Speech: A Dataset for Identifying Hate Instigating Speech}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2023},
  doi = {10.18653/v1/2023.findings-emnlp.412},
}
Figure