TL;DR
Automated hate speech detection often stumbles when hate is implicit or masked within internet memes. This paper tackles this challenge by introducing a high-quality, expertly annotated dataset of 5,000 multimodal tweets and demonstrating that while context-aware text models excel at identifying implicit text, multimodal models achieve the best overall performance by significantly reducing false positives.
What Problem Does This Paper Address?
Most automated hate speech detection systems are designed to catch explicit abuse, such as slurs or direct threats. However, bad actors frequently evade these systems using implicit strategies like "dog whistling"—employing coded, intentionally ambiguous language that only a specific target audience understands.
This problem is amplified in multimodal content like memes, where a benign image paired with benign text can combine to create a deeply hateful message (a phenomenon known as modal dissonance). Existing datasets often lack the granular, high-quality annotations needed to train and evaluate machine learning models on these subtle, cross-modal nuances.
Methods
The authors evaluate how different levels of semantic and visual context affect model performance by benchmarking four distinct model classes:
- Unimodal-Image: An Xception network pre-trained on ImageNet to evaluate visual-only signals.
- Context-Invariant Unimodal-Text: A baseline bidirectional LSTM (Bi-LSTM) using pre-trained fastText word embeddings.
- Context-Aware Unimodal-Text: A transformer-based model (RoBERTa-large) capable of evaluating words dynamically based on their surrounding textual context.
- Multimodal: A Multimodal Bitransformer (MMBT) architecture that fuses image embeddings (from a ResNet-152 backbone) with tokenized text embeddings via a bidirectional transformer attention mechanism to capture the interaction between text and imagery.
Datasets
The authors introduced a newly annotated dataset of 5,000 multimodal tweets sampled from the existing MMHS150K dataset. To ensure data quality over sheer scale, they eschewed standard crowdsourcing (like Amazon Mechanical Turk) in favor of experienced, rigorously trained expert annotators.
The dataset features a hierarchical schema that labels:
- Primary Attribute: Hate, Counterspeech, Reclaimed, or Neutral.
- Modality: Which mode (text, image, or both) carried the hateful message.
- Strategy: Whether the hate was Explicit or Implicit. Implicit hate was further categorized into Pseudo-factual, Normative, Coded, or Creative expressions.
Key Insights and Ideas
- The False Positive Trade-off: Surprisingly, the text-only transformer model (RoBERTa) was the most accurate at classifying the specific subtask of implicit hate. However, the multimodal model won out overall because it was significantly better at identifying non-hateful content, thereby lowering the rate of false positives.
- Anchoring Ambiguity: Human annotators frequently disagree on implicit and multimodal content. When evaluating performance on "ambiguous" data (where human annotators disagreed), the multimodal model handled the confusion best. This suggests that cross-modal context helps anchor and clarify ambiguous text.
- Human vs. Model Errors: A qualitative error analysis revealed that 25% of the multimodal model's "errors" occurred because the model actually classified the tweet better than the human annotators did, highlighting the extreme difficulty of manual annotation in this domain.
Key Results
- Overall Performance: The multimodal model achieved the highest overall F1-score of 0.771, outperforming the context-aware text model (0.754), context-invariant text model (0.719), and the image-only model (0.544).
- Strategy Breakdown: For the specific task of identifying implicit hate, the context-aware text model reached an accuracy of 0.941 compared to the multimodal model's 0.899. However, the multimodal model excelled at identifying non-hateful content (0.732 accuracy vs. RoBERTa's 0.678).
- Handling Ambiguity: On data with full annotator agreement (unambiguous), RoBERTa led with an F1 of 0.875. On ambiguous data, the multimodal model achieved the top F1-score of 0.692.
Limitations and Future Work
- Data Collection Bias: The initial dataset relied on text-based query filtering via the Twitter API, which inherently biases the dataset toward text-heavy features rather than purely visual or organically discovered memes.
- Modal Dissonance Penalties: In some cases, the multimodal model suffered from false negatives because a benign image diluted a strongly hateful text signal, indicating that current fusion mechanisms still struggle to balance conflicting cross-modal cues perfectly.
- Lack of Speaker Context: The models frequently misclassified "reclaimed" language (e.g., minority communities using slurs self-referentially) because the system lacked external context regarding the speaker's identity. Incorporating user metadata remains an open challenge for future work.
Summary updated