TL;DR
Internet Memes (IMs) are a rich form of online communication, but their complex, multimodal nature has made them difficult to analyze systematically. This paper introduces IMKG, the first comprehensive knowledge graph for internet memes, which captures their text, vision, and metadata to enable deeper understanding and novel applications.
What Problem Does This Paper Address?
Internet memes are more than just funny pictures with captions; they are a cultural phenomenon that reflects and shapes online discourse. However, their inherent characteristics—multimodality (text + image), succinctness (complex ideas in simple forms), relatability (community-specific understanding), and fluidity (constant evolution and variation)—make them challenging to analyze at scale. Previous research has often focused on isolated aspects like spread or hate speech detection, neglecting the deeper semantic structure. This paper aims to bridge this gap by building a knowledge graph that explicitly represents the multifaceted semantics of IMs, leveraging Semantic Web technologies.
Methods
The authors construct the Internet Meme Knowledge Graph (IMKG) through a four-step pipeline:
- Data Modelling: They design a conceptual model to capture key IM properties, including
Media Frame(the original scene),Meme(an instance of a Media Frame with a caption), andTemplate(a reusable structure for generating memes). This model aims to represent multimodality, succinctness, relatability, and fluidity. - Data Collection: IMKG is bootstrapped from three primary sources:
- Wikidata: Used as a seed for existing meme entities and background knowledge.
- Know Your Meme (KYM): A collaborative encyclopedia providing rich information on meme lore, origins, and variations, scraped from their website.
-
ImgFlip: A popular meme generation website, used to collect meme templates and user-generated memes. Social media platforms were excluded due to access restrictions and limited contextual information.
-
Data Enrichment: The collected data is enriched through:
- Textual Enrichment: Extracting and linking entities from KYM descriptions and ImgFlip captions to Wikidata.
- Visual Enrichment: Using Google Vision API for object detection on media frame images, mapping detected entities to Wikidata.
- Knowledge-Based Enrichment: Iteratively incorporating background knowledge from Wikidata about memes and their entities.
- Data Integration: Data from various sources and formats (JSON, KGTK TSV) is converted and integrated into a cohesive graph. They use RML for mapping and publish IMKG in both RDF and a labeled property graph (LPG) format to support different analytical needs.
Datasets
- Wikidata: Used as a source of seed memes and background knowledge.
- Know Your Meme (KYM): A large corpus of internet memes, their lore, and origins, scraped from the website.
- ImgFlip: A dataset of meme templates and user-generated memes.
The paper introduces IMKG itself as a novel, publicly available resource.
Key Insights and Ideas
- Semantic Web for Memes: The paper demonstrates that Semantic Web technologies (RDF, knowledge graphs) are well-suited to capture the complex, layered semantics of internet memes.
- Multimodal Representation: IMKG explicitly models the interplay between visual (media frame) and textual (caption) components, recognizing that both are crucial for understanding meme meaning.
- Stratified Semantics: The model distinguishes between the original
Media Frame, the specificMemeinstance, and the reusableTemplate, providing a structured way to understand meme evolution and variation. - Enrichment is Key: Integrating data from diverse sources (encyclopedias, generators, KGs) and enriching it with entity linking and background knowledge significantly enhances the graph's utility and enables novel queries.
- Relatability as a Core Feature: The paper highlights how relatability, often tied to community-specific lore and shared cultural references, is a fundamental aspect of meme success and understanding.
Key Results
- IMKG contains over 4.8 million nodes and 16.5 million edges, integrating data from KYM, ImgFlip, and Wikidata.
- Over 20% of IMKG's edges connect memes to entities extracted from images or text, and over 3% come from Wikidata enrichment.
- The graph enables novel use cases, such as querying for IMs based on specific films (e.g., 130 media frames depicting Sponge Bob) or identifying the most "meme-able" persons.
- Analysis shows that Wikidata's background knowledge plays a crucial role in connecting memes and enriching their semantic context.
- The paper demonstrates that combining structural graph information with content similarity (via SentenceBERT embeddings) can reveal meaningful relationships between memes.
Limitations and Future Work
- Limitations:
- The current IMKG primarily covers "Image macros" and excludes other meme formats like GIFs and videos.
- While social media platforms were excluded due to access issues, their inclusion would further enhance coverage.
- The current model doesn't fully leverage the rich textual content within paragraphs, which could be better encoded using language models.
- Future Work:
- Expand coverage to include video memes and other modalities.
- Incorporate user profiles and community data to better model relatability.
- Integrate FrameNet for richer semantic analysis of meme constructions.
- Develop novel reasoning tasks, such as meme-based Question Answering (QA).
- Plan for recurrent releases of the KG to manage scalability and evolving data.
- Incorporate IMKG into neuro-symbolic methods for improved meme analysis and explainability.
Summary updated