Multimodal Detection of Harmful and Misogynistic Memes

22 concepts23 papers1 proposal

Internet memes have evolved from lighthearted insider jokes into a dominant form of online commentary. However, their unique format—the juxtaposition of an image and a snippet of text—has also made them a preferred vector for spreading harmful content, including trolling, hate speech, and misogyny.

Detecting harmful memes is a notoriously difficult challenge for traditional content moderation systems. The core difficulty lies in the fact that the harm is rarely contained within the text alone or the image alone. Instead, the malicious meaning is emergent, arising purely from the interaction between the two modalities. To combat this, researchers have developed multimodal deep learning frameworks designed to align, fuse, and interpret visual and textual signals simultaneously.

The Challenge of Emergent Semantics

Why do standard text classifiers or computer vision models fail when evaluating memes? In many harmful memes, the individual components appear entirely benign when isolated. For example, an image might depict a harmless scene from a movie, and the text might be a commonplace phrase. However, when overlaid, the text recontextualizes the image (or vice versa) to produce a derogatory, hateful, or misogynistic message.

As Gu et al. observe, the ease of meme composition allows bad actors to subtly embed harmful messages under the guise of humor or irony. This camouflage exploits the semantic gap between what a model "sees" and what it "reads." While text often carries the primary offensive weight in a meme—a phenomenon quantified by Elkhatib et al. in a controlled study—relying solely on text analysis misses the satirical or targeted framing provided by the visual asset. Automated systems must therefore possess the capability to analyze the nuances of internet culture to prevent this content from proliferating (Bejan).

Architectures for Multimodal Fusion

To capture these emergent semantics, researchers design architectures that bridge the gap between vision and language. The evolution of these models reflects a progression from simple feature concatenation to deeply integrated joint representations.

Early and Late Fusion Baselines

Early approaches often relied on training separate feature extractors for each modality and combining them at the decision layer. For instance, Shaun H et al. combined a Multinomial Naive Bayes model for text processing with a ResNet50 model for image classification. Similarly, Huang et al. fused a Bidirectional Gated Recurrent Unit (BiGRU) for text with a Convolutional Neural Network (CNN) for images. While these models establish solid baselines, they often fail when the relationship between text and image is highly abstract or metaphorical.

Cross-Modal Attention and Transformers

To capture finer interactions, modern frameworks utilize attention mechanisms that allow the text tokens to "attend" to specific regions of an image, and vice versa. Hegde et al. demonstrated the power of a transformer-transformer architecture, which focuses attention on salient features across both modalities simultaneously. By leveraging pre-trained language models alongside visual transformers, systems can identify subtle correlations that simpler fusion techniques miss (Rao et al.).

[ Image Input ] ---> [ Vision Transformer / CNN ] ---\
                                                      +---> [ Cross-Modal Attention / Fusion ] ---> [ Harm Classification ]
[ Text Input ]  ---> [ Language Transformer ] -------/

Advanced Representation Learning

Beyond standard classification heads, researchers use sophisticated modeling paradigms to map the shared space:

  • Variational Autoencoders (VAEs): Gu et al. introduced MMVAE, a multi-modal, multi-task VAE designed to learn a robust joint distribution of visual and textual features, outperforming unimodal baselines by capturing underlying latent factors of misogyny.
  • Graph Neural Networks (GNNs): Rather than treating features as a flat vector, Harshvardhan Srivastava leveraged graph networks to model the relationships between entities in the meme, enabling precise multi-label classification of specific misogyny types.
  • Multi-Task and Ensemble Learning: Tao et al. demonstrated that sharing knowledge between related subtasks (e.g., identifying sarcasm while simultaneously detecting hate speech) via ensemble methods significantly boosts overall detection accuracy.

The challenge of meme detection intensifies when moving outside of English into low-resource or highly localized linguistic environments. A significant body of recent work focuses on Dravidian languages (such as Tamil and Malayalam) and Bangla, where harmful content frequently leverages code-mixing (blending native scripts with English) and deep cultural metaphors.

In these settings, standard multilingual models often fall short. To address this, Ratul et al. paired specialized language models like IndicBERT with ResNet architectures to capture regional linguistic nuances. Chauhan et al. deployed transformer-based multimodal frameworks specifically tuned to handle the subtle misogyny embedded in code-mixed text.

Furthermore, the mathematical boundaries for classifying these complex, low-resource inputs are being reimagined. Arunachalam et al. introduced the use of Kolmogorov-Arnold Networks (KANs)—such as Chebyshev KAN and FastKAN—as alternatives to traditional Multi-Layer Perceptrons (MLPs) for multimodal fusion, finding that they offer superior adaptability when learning the intricate decision boundaries required for Tamil and Malayalam meme classification.

Beyond Classification: Explaining the Metaphor

The cutting edge of this research direction moves away from binary "harmful/safe" labels toward interpretability. Because memes rely heavily on cultural shorthand, a model that merely flags a meme without explanation is difficult for human moderators to verify.

To bridge this gap, Mia et al. introduced the BANMIME dataset for Bangla memes, pioneering the use of Chain-of-Thought (CoT) fine-tuning on Vision-Language Models (VLMs). Instead of outputting a raw prediction, these models are trained to reason through the meme's visual and textual metaphors step-by-step, generating a natural language explanation of why a specific image-text combination constitutes misogyny. This shift toward explainable AI ensures that automated content moderation can be both highly accurate and transparently accountable.

Key Foundational Work

Detecting Misogynous Tweets (2018) — contributed 1 concept elements to this direction.
MemoSYS at SemEval-2020 Task 8: Multimodal Emotion Analysis in Memes (2020) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_spread_of_harmful_online_content_9cabf78a,
  title = {spread of harmful online content},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/spread-of-harmful-online-content/rdr_9cabf78ae1e243e9876afaf9436f9add},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_spread_of_harmful_online_content_9cabf78a_v1,
  title = {spread of harmful online content},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/spread-of-harmful-online-content/rdr_9cabf78ae1e243e9876afaf9436f9add/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake