Human-in-the-Loop Hate Speech Classification in a Multilingual Context

Ana Kotarcic, Dominik Hangartner, Fabrizio Gilardi, Selina Kurer, Karsten DonnayEMNLP 2022

TL;DR

This paper introduces a novel human-in-the-loop (HITL) BERT-based pipeline for multilingual hate speech classification, specifically tailored for the complex Swiss context. It demonstrates that continuous human oversight and iterative retraining are crucial for maintaining robust performance in real-world, dynamic online environments, outperforming existing multilingual classifiers.

What Problem Does This Paper Address?

The proliferation of online hate speech poses a significant challenge, yet existing classification systems often fall short. Many studies focus on single languages (primarily English) and neglect crucial aspects like post-deployment performance, ongoing classifier maintenance, and infrastructural limitations. This is particularly problematic for smaller organizations and regional contexts with unique linguistic and cultural nuances, like Switzerland, where existing tools fail to capture context-specific hate speech. The authors aim to bridge this gap by developing a sustainable, multilingual HITL classification system that addresses these real-world deployment challenges.

Methods

The core of the paper is a human-in-the-loop (HITL) BERT-based hate speech classification pipeline. This system integrates an automatic retraining process with human oversight. Initially, a BERT model (mBERT) is trained on a dataset. Its predictions (weak labels) are then reviewed and corrected by human annotators, generating strong labels. These corrected examples are fed back into the training set, leading to iterative retraining of the model. This active learning approach allows the model to adapt to evolving language and new hate speech patterns. For classification, they experimented with multinomial Naïve Bayes (MNB) and various BERT architectures, ultimately selecting mBERT for its multilingual capabilities.

Datasets

The study utilizes the Swiss Hate Speech Corpus, a newly compiled dataset specifically for this research. It comprises over 422,000 unique annotations derived from three main sources:

  1. Comments collected by an NGO partner.
  2. Online newspaper comments from three Swiss national outlets (ON1, ON2, ON3).
  3. Tweets from "politically interested users" via the Twitter API.

The corpus is multilingual, primarily featuring (Swiss) German and French, and includes annotations for hate speech, toxic speech, and up to 10 multilabel target groups.

Key Insights and Ideas

  • Human-in-the-Loop is Non-Negotiable for Real-World Robustness: The paper strongly emphasizes that static, one-time-trained models are insufficient. Continuous human involvement is vital for adapting to the dynamic nature of online discourse and maintaining classifier performance over time.
  • Context is King (Especially in Multilingual Settings): The Swiss context, with its unique linguistic, sociocultural, and political landscape, highlights the limitations of generic classifiers. Developing context-aware models is crucial.
  • Post-Deployment Evaluation is as Important as Pre-Deployment: Testing classifiers in their actual deployment environment reveals performance drops (drift) that are not apparent in lab settings, underscoring the need for ongoing monitoring and retraining.
  • Toxic Speech vs. Hate Speech Nuance: The study found that treating toxic speech as a separate category (rather than conflating it with hate speech or dropping it entirely) led to better performance for hate speech detection, suggesting a need for careful definition and annotation.

Key Results

  • The developed HITL mBERT classifier achieved an F1 score of 80.5, outperforming the current best-performing BERT-based multilingual classifier by 5.8 F1 points in German and 3.6 F1 points in French.
  • Cross-platform and cross-dataset experiments showed a significant performance drop (up to 14.2 F1 points) when models trained on one platform were tested on another, highlighting the challenge of generalization.
  • Temporal drift was observed, with performance decreasing by 14.3 F1 points when prediction sets were six months apart from training sets.
  • The analysis of hate speech probability thresholds suggested an optimal cutoff of 0.90 to guarantee optimal model performance.

Limitations and Future Work

  • Infrastructural Limitations: The authors faced significant challenges with cloud services and university cluster resources, limiting scalability and requiring workarounds for data parallelization and hyperparameter tuning.
  • Data Bias: The dataset, while diverse, is prone to biases stemming from sources, annotation processes, pre-trained models, and the prevalence of certain target groups.
  • Inter-Annotator Agreement: Low inter-rater reliability (0.3-0.4) was observed, attributed to the subjective and challenging nature of hate speech annotation, especially with implicit statements and humor.
  • Future Work: The authors suggest exploring more advanced data balancing techniques, implementing multilabel classifiers for better target group representation, and developing more robust infrastructural frameworks for easier deployment and long-term maintenance of AI systems. They also advocate for collaborative efforts between software companies and research institutions to improve model parallelization and cloud service compatibility with data protection requirements.

Summary updated

BibTeX
@inproceedings{kotarcic2022human,
  author = {Ana Kotarcic and Dominik Hangartner and Fabrizio Gilardi and Selina Kurer and Karsten Donnay},
  title = {{Human-in-the-Loop Hate Speech Classification in a Multilingual Context}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2022},
}
Figure