Detecting Hate Speech and Misogyny in Multimodal Memes

56 concepts56 papers2 proposals

Internet memes have evolved into a primary vehicle for digital communication, but their unique format makes them highly effective tools for spreading hate speech, misogyny, and propaganda. Traditional content moderation systems rely heavily on text-based natural language processing or isolated image classification. However, these unimodal approaches consistently fail when applied to memes.

The core difficulty lies in emergent semantics: the harmful message is rarely contained within a single modality. Instead, the offense arises entirely from the non-linear interplay between an otherwise benign image and a seemingly harmless caption.

The Intuition of Cross-Modal Incongruity

To understand why memes evade standard classifiers, consider a meme containing a stock photo of a kitchen paired with a caption about corporate leadership. Individually, an image classifier sees a domestic setting, and a text classifier reads a standard professional discussion. Neither flags any violation. However, when juxtaposed, the combination enforces a degrading gender stereotype.

As highlighted by Singh et al. in “Female Astronaut: Because sandwiches won’t make themselves up there”: Towards Multimodal misogyny detection in memes, the individual modalities in harmful memes are frequently free of explicit hate speech. The toxicity is implicit, relying heavily on irony, sarcasm, and cultural tropes. Because the visual and textual cues modify or invert each other's literal meanings, a model cannot simply sum the probabilities of each modality; it must model their joint semantic space.

Benchmarking the Multimodal Threat

To address this challenge, researchers have constructed specialized benchmarks targeting various dimensions of multimodal abuse:

The Evolution of Fusion Architectures

The core engineering problem in this domain is fusion: how do we force a model to read text and view an image simultaneously, rather than sequentially?

Early Feature Concatenation (Late Fusion)

Early architectures extracted visual features using a Convolutional Neural Network (CNN) and textual features using a Recurrent Neural Network or a simple Transformer, subsequently concatenating the resulting vectors before classification. For example, Huang et al. in HUB@DravidianLangTech-EACL2021: Meme Classification for Tamil Text-Image Fusion combined a BiGRU for text with a CNN for images.

However, as noted by Hasan et al. in CUET-NLP@DravidianLangTech-ACL2022: Investigating Deep Learning Techniques to Detect Multimodal Troll Memes, these shallow fusion strategies often fail to outperform strong text-only baselines. Because the features are processed independently until the final layer, the model misses the step-by-step contextual interaction between words and visual regions.

Vision-Language Alignment and Cross-Attention

To overcome the limitations of late fusion, researchers pivoted toward models that perform early or mid-stage cross-modal alignment. Two main approaches dominate:

  1. Cross-Modal Attention: Models like ViLBERT and VisualBERT use transformer layers where visual region embeddings and textual token embeddings attend directly to one another. Lorentz et al. in INF-UFRGS at SemEval-2022 Task 5: analyzing the performance of multimodal models demonstrated that ViLBERT's co-attentional mechanism allows the model to dynamically weigh visual elements based on specific words in the caption, making it highly effective at catching subtle online hate speech.
  2. Contrastive Joint Embeddings: Contrastive Language-Image Pre-training (CLIP) maps both images and text into a shared semantic space. Arango et al. in HateU at SemEval-2022 Task 5: Multimedia Automatic Misogyny Identification leveraged CLIP to determine whether the visual context aligns with or subverts the textual transcription, capturing the holistic meaning of the meme.

Generative Reasoning and Explanatory AI

The latest paradigm shifts away from black-box classification toward generative explanation. Modern frameworks utilize Large Multimodal Models (LMMs) and Chain-of-Thought (CoT) prompting to explicitly unpack the metaphors embedded within memes. Instead of outputting a simple binary label, models are trained to localize the metaphor and explain why it constitutes abuse, as seen in the BANMIME framework (Mia et al.).

Despite these advancements, the problem remains far from solved. Comprehensive evaluations on benchmarks like GOAT-Bench, introduced by Luo et al. in GOAT-Bench : Safety Insights to Large Multimodal Models through Meme-Based Social Abuse, reveal that even state-of-the-art LMMs remain highly vulnerable to sophisticated, implicit social abuse when it is masked by internet humor and complex visual-textual metaphors.

Key Foundational Work

Findings of the Shared Task on Troll Meme Classification in Tamil (2021) — contributed 1 concept elements to this direction.
NLP-CUET@DravidianLangTech-EACL2021: Investigating Visual and Textual Features to Identify Trolls from Multimodal Social Media Memes (2021) — contributed 1 concept elements to this direction.
BERT at SemEval-2020 Task 8: Using BERT to Analyse Meme Emotions (2020) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_multimodal_hate_speech_dataset_65c3df5a,
  title = {multimodal hate speech dataset},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/multimodal-hate-speech-dataset/rdr_65c3df5a3d694f8cbe0b3ec7f059bc6b},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_multimodal_hate_speech_dataset_65c3df5a_v1,
  title = {multimodal hate speech dataset},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/multimodal-hate-speech-dataset/rdr_65c3df5a3d694f8cbe0b3ec7f059bc6b/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake