Deciphering Implicit Hate: Evaluating Automated Detection Algorithms for Multimodal Hate

Austin Botelho, Bertie Vidgen, Scott A. Hale

TL;DR

Automated hate speech detection often stumbles when hate is implicit or masked within internet memes. This paper tackles this challenge by introducing a high-quality, expertly annotated dataset of 5,000 multimodal tweets and demonstrating that while context-aware text models excel at identifying implicit text, multimodal models achieve the best overall performance by significantly reducing false positives.

What Problem Does This Paper Address?

Most automated hate speech detection systems are designed to catch explicit abuse, such as slurs or direct threats. However, bad actors frequently evade these systems using implicit strategies like "dog whistling"—employing coded, intentionally ambiguous language that only a specific target audience understands.

This problem is amplified in multimodal content like memes, where a benign image paired with benign text can combine to create a deeply hateful message (a phenomenon known as modal dissonance). Existing datasets often lack the granular, high-quality annotations needed to train and evaluate machine learning models on these subtle, cross-modal nuances.

Methods

The authors evaluate how different levels of semantic and visual context affect model performance by benchmarking four distinct model classes:

  1. Unimodal-Image: An Xception network pre-trained on ImageNet to evaluate visual-only signals.
  2. Context-Invariant Unimodal-Text: A baseline bidirectional LSTM (Bi-LSTM) using pre-trained fastText word embeddings.
  3. Context-Aware Unimodal-Text: A transformer-based model (RoBERTa-large) capable of evaluating words dynamically based on their surrounding textual context.
  4. Multimodal: A Multimodal Bitransformer (MMBT) architecture that fuses image embeddings (from a ResNet-152 backbone) with tokenized text embeddings via a bidirectional transformer attention mechanism to capture the interaction between text and imagery.

Datasets

The authors introduced a newly annotated dataset of 5,000 multimodal tweets sampled from the existing MMHS150K dataset. To ensure data quality over sheer scale, they eschewed standard crowdsourcing (like Amazon Mechanical Turk) in favor of experienced, rigorously trained expert annotators.

The dataset features a hierarchical schema that labels:

  • Primary Attribute: Hate, Counterspeech, Reclaimed, or Neutral.
  • Modality: Which mode (text, image, or both) carried the hateful message.
  • Strategy: Whether the hate was Explicit or Implicit. Implicit hate was further categorized into Pseudo-factual, Normative, Coded, or Creative expressions.
Examples of tweets mapped to the authors' implicit hate annotation strategies.

Key Insights and Ideas

  • The False Positive Trade-off: Surprisingly, the text-only transformer model (RoBERTa) was the most accurate at classifying the specific subtask of implicit hate. However, the multimodal model won out overall because it was significantly better at identifying non-hateful content, thereby lowering the rate of false positives.
  • Anchoring Ambiguity: Human annotators frequently disagree on implicit and multimodal content. When evaluating performance on "ambiguous" data (where human annotators disagreed), the multimodal model handled the confusion best. This suggests that cross-modal context helps anchor and clarify ambiguous text.
  • Human vs. Model Errors: A qualitative error analysis revealed that 25% of the multimodal model's "errors" occurred because the model actually classified the tweet better than the human annotators did, highlighting the extreme difficulty of manual annotation in this domain.

Key Results

  • Overall Performance: The multimodal model achieved the highest overall F1-score of 0.771, outperforming the context-aware text model (0.754), context-invariant text model (0.719), and the image-only model (0.544).
  • Strategy Breakdown: For the specific task of identifying implicit hate, the context-aware text model reached an accuracy of 0.941 compared to the multimodal model's 0.899. However, the multimodal model excelled at identifying non-hateful content (0.732 accuracy vs. RoBERTa's 0.678).
  • Handling Ambiguity: On data with full annotator agreement (unambiguous), RoBERTa led with an F1 of 0.875. On ambiguous data, the multimodal model achieved the top F1-score of 0.692.

Limitations and Future Work

  • Data Collection Bias: The initial dataset relied on text-based query filtering via the Twitter API, which inherently biases the dataset toward text-heavy features rather than purely visual or organically discovered memes.
  • Modal Dissonance Penalties: In some cases, the multimodal model suffered from false negatives because a benign image diluted a strongly hateful text signal, indicating that current fusion mechanisms still struggle to balance conflicting cross-modal cues perfectly.
  • Lack of Speaker Context: The models frequently misclassified "reclaimed" language (e.g., minority communities using slurs self-referentially) because the system lacked external context regarding the speaker's identity. Incorporating user metadata remains an open challenge for future work.

Summary updated

Abstract

Accurate detection and classification of online hate is a difficult task. Implicit hate is particularly challenging as such content tends to have unusual syntax, polysemic words, and fewer markers of prejudice (e.g., slurs). This problem is heightened with multimodal content, such as memes (combinations of text and images), as they are often harder to decipher than unimodal content (e.g., text alone). This paper evaluates the role of semantic and multimodal context for detecting implicit and explicit hate. We show that both text- and visual- enrichment improves model performance, with the multimodal model (0.771) outperforming other models' F1 scores (0.544, 0.737, and 0.754). While the unimodal-text context-aware (transformer) model was the most accurate on the subtask of implicit hate detection, the multimodal model outperformed it overall because of a lower propensity towards false positives. We find that all models perform better on content with full annotator agreement and that multimodal models are best at classifying the content where annotators disagree. To conduct these investigations, we undertook high-quality annotation of a sample of 5,000 multimodal entries. Tweets were annotated for primary category, modality, and strategy. We make this corpus, along with the codebook, code, and final model, freely available.
BibTeX
@inproceedings{botelho2021deciphering,
  author = {Austin Botelho and Bertie Vidgen and Scott A. Hale},
  title = {{Deciphering Implicit Hate: Evaluating Automated Detection Algorithms for Multimodal Hate}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
}

Loading related content...

Figure