TL;DR
This paper tackles the challenge of identifying who is being targeted by hate speech online, not just that hate speech is occurring. They developed and evaluated neural network models that can predict multiple targeted identity groups and sub-groups simultaneously, showing promising generalizability across different datasets. This work is crucial for a deeper understanding and more effective mitigation of online hate speech.
What Problem Does This Paper Address?
While significant effort has gone into detecting hate speech itself, understanding who is being targeted is a less explored but equally critical area. Hate speech is defined by its targeting of individuals or groups based on their identity. Without accurately identifying these targets, it's difficult to analyze the impact of hate speech, develop targeted interventions, or even fully understand the scope of the problem. Existing work often focuses on binary classifications (e.g., racism vs. sexism) or individual vs. group targets. This paper aims to bridge this gap by developing models that can predict a broad range of targeted identity groups and their sub-groups within online text, moving beyond simple categories to a more nuanced understanding.
Methods
The core of this research involves training and evaluating transformer-based neural network models for multi-label binary prediction. This means for each comment, the model predicts the presence or absence of multiple identity group targets simultaneously. They experimented with various pre-trained transformer architectures, including Universal Sentence Encoder, BERT, and RoBERTa, and explored different hyperparameter configurations. A key methodological choice was using "soft labels" during training, where the proportion of annotators identifying a group as a target informed the model, and then using majority voting for evaluation. This approach acknowledges the inherent subjectivity and potential disagreement among human annotators.
Datasets
The primary dataset used for training and internal validation was the Measuring Hate Speech (MHS) corpus. This corpus contains approximately 50,000 social media comments with annotations for 8 broad identity groups and 12 sub-groups, collected from Reddit, YouTube, and Twitter.
For external validation and generalization testing, two other corpora were used:
- HateCheck Corpus: A set of synthetically generated "functional tests" for hate speech detection models, designed to be a sanity check.
- Gab Hate Corpus (GHC): A dataset of 27,665 posts from the social media platform Gab, annotated for hate-based rhetoric and target identity groups.
Key Insights and Ideas
- Nuance in Targeting: The paper highlights that "targeting" in hate speech can be complex, encompassing both hateful and supportive speech, and that datasets may define it differently, impacting model generalization.
- Intersectionality Matters: The authors emphasize the importance of distinguishing between intersectional targets (e.g., "Black women") and multiple distinct targets (e.g., "Black people" and "women" separately), suggesting future work should develop labeling schemes to capture this.
- Data Imbalance is a Challenge: Performance on less represented identity groups (e.g., age, disability) significantly lags behind more common targets, underscoring the need for better data collection and augmentation strategies.
- Annotator Subjectivity: The moderate agreement among annotators (Krippendorff's alpha values) indicates that identifying targeted identity groups is not always straightforward, suggesting that approaches incorporating this disagreement (like soft labels) are valuable.
Key Results
The models demonstrated strong predictive performance on the MHS corpus, achieving ROC AUCs above 0.90 for all identity groups and PR AUCs above 0.80 for the most frequent groups (gender, race, religion, sexual orientation). Performance generally tracked with the incidence rate of the identity group, meaning more common targets were predicted more accurately.
- Broad Identity Groups: F1 scores ranged from 0.7 to 0.85 for the most frequent groups.
- Identity Sub-Groups: Performance on sub-groups was generally lower than for broad groups, with racial identities showing better performance than gender identities. For example, Black-targeting speech achieved a median F1 score of 0.72.
- Generalization: Models generalized well to the HateCheck corpus, showing accuracies of 0.97-0.99 and F1 scores above 0.90. Performance on the Gab Hate Corpus was also good but showed a slight drop, particularly for less represented groups like disability and national origin.
- Multi-Target Performance: Performance on comments targeting multiple identity groups varied, generally being better for pairs with more samples (e.g., gender, race) but also showing interesting patterns for less related groups.
Limitations and Future Work
- Definition of "Target": The paper acknowledges that the definition of "target" can vary across datasets, impacting generalization. Future work could standardize these definitions and develop measurement scales for "targeting."
- Intersectional vs. Multiple Targets: The current labeling scheme doesn't distinguish between intersectional targets and multiple distinct targets. Future research should focus on developing labeling instruments that can capture this crucial distinction, especially for understanding intersectional identity.
- Multi-Task Learning for Sub-Groups: The authors suggest exploring multi-task networks that can predict multiple sets of sub-groups simultaneously, potentially improving performance on intersectional targets.
- Annotator Disagreement: The subjectivity in annotator agreement suggests exploring data perspectivist approaches and developing more structured annotation instruments to understand and potentially reduce disagreement.
- Qualitative Analysis: Further qualitative examination of comments and annotator reasoning could provide deeper insights into the concept of "targeting" and improve model interpretability.
Summary updated