Computational Meme Understanding: A Survey

Khoi P. N. Nguyen, Vincent NgEMNLP 2024 · DOI

Introduction

Internet memes have evolved from simple digital jokes into a sophisticated, multimodal language of the modern web. They serve as vehicles for humor, social commentary, and, increasingly, the dissemination of hate speech and misinformation. Computational Meme Understanding (CMU) is an emerging field that seeks to automate the comprehension of these complex artifacts. This survey provides a comprehensive overview of the field, establishing a formal taxonomy and evaluating the current state-of-the-art in AI-driven meme analysis.

A Multidimensional Taxonomy

A central challenge in CMU is the sheer variety of meme formats. The authors propose a three-dimensional taxonomy to organize this space:

  1. Forms: Memes are categorized by their physical structure. Remixed Images involve the manipulation of existing templates (e.g., "Macros" or "Shops"), while Stable Images are unaltered photos or screenshots repurposed for a new context.
  2. Functions: This dimension looks at the "illocutionary act"—what the meme is intended to achieve, such as mocking, praising, or stereotyping.
  3. Topics: Memes are often tied to specific semantic themes, which are frequently time-sensitive, such as global health crises or political elections.
Figure 1: A formal taxonomy of meme forms, distinguishing between remixed and stable images.

Core Tasks in CMU

The research community generally approaches meme understanding through three primary technical lenses:

  • Classification: The most mature task, involving the assignment of labels like "hateful" or "sarcastic."
  • Interpretation: A more generative task where models must produce a natural language "final message" that captures the meme's underlying intent.
  • Explanation: This requires models to provide a textual justification for their labels, explaining why a certain entity is being targeted or why a specific sentiment is present.

The Data Landscape

The survey evaluates 24 key datasets that drive progress in the field. Benchmarks like HatefulMemes focus on binary classification of malicious content. More advanced datasets like MemeCap challenge models to interpret visual metaphors, while HatReD and ExHVV provide the ground-truth text necessary for training models to explain their reasoning. While English remains the dominant language, there is a growing push toward multilingual datasets to capture the global nature of meme culture.

The Role of Context and Reasoning

The primary bottleneck in CMU is what the authors term "Subcultural Literacy." Unlike standard image recognition, memes are "more than the sum of their parts." Understanding them requires access to a vast, evolving database of internet culture, slang, and current events.

Temporal context is particularly critical. A meme's meaning can shift dramatically over time. For instance, a political meme created during a specific news cycle might be interpreted as a factual claim at the time, but as a satirical or contradictory statement just a few years later.

Figure 2: An example of a 'stacked' meme requiring temporal and political context for correct interpretation.

Furthermore, most memes follow a "Macro" logic, where a base template carries a traditional meaning that is modified by new text. AI systems must not only recognize the template but also understand how the new overlay interacts with the template's established cultural baggage.

Results and Future Directions

While classification models are reaching high levels of accuracy (often exceeding 90% on binary tasks), complex reasoning remains a significant hurdle. In tasks involving the detection of subtle persuasion techniques, even the best models struggle, often failing to reach an F1 score of 0.60. Human evaluations of model-generated explanations show a "correctness gap," with models frequently hallucinating details or failing to identify key visual cues.

Future work must move beyond static images to address the complexities of GIFs and short-form video. Additionally, models need mechanisms for active knowledge acquisition—the ability to "look up" new trends on platforms like Know Your Meme—to keep pace with the rapid evolution of digital communication.

Summary updated

BibTeX
@inproceedings{nguyen2024computational,
  author = {Khoi P. N. Nguyen and Vincent Ng},
  title = {{Computational Meme Understanding: A Survey}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  doi = {10.18653/v1/2024.emnlp-main.1184},
}
Figure