VIDA: The Visual Incel Data Archive. A Theory-oriented Annotated Dataset To Enhance Hate Detection Through Visual Culture

Selenia Anastasi, Florian Schneider, Chris Biemann, Tim FischerNAACL 2024

TL;DR

This paper introduces VIDA, a novel multimodal dataset of images and memes from online Incel communities, designed to improve hate speech detection. By developing a theory-driven annotation framework that considers content, form, and stance, the authors provide a valuable resource for understanding and combating visual extremism.

What Problem Does This Paper Address?

The proliferation of online content, particularly visual media like images and memes, has made it increasingly challenging to detect and understand extremist ideologies. While text-based hate speech has received considerable attention, the visual dimension of extremism, especially within niche online communities, remains underexplored. Machine learning models struggle to grasp the nuanced, often implicit, meanings embedded in visual culture, which can be crucial for identifying offensive content and understanding the cultural context of hate speech. This paper addresses this gap by creating a specialized dataset and annotation framework to better analyze visual extremism, focusing on the Incelosphere, a community known for its misogynistic and hateful rhetoric.

Methods

The authors developed a new taxonomy for annotating visual content, inspired by Shifman's (2014) work on memes. This taxonomy operates on three levels:

  • Content: The main topic, idea, and ideology conveyed by an image.
  • Form: The physical characteristics of the image, including its shape, morphological dimension, and genre.
  • Stance: The tone and style of communication, specifically focusing on the sender's hostile position and how it's encoded within the image or its accompanying text.

This framework was applied to a dataset collected from Italian and Anglophone Incel forums. The annotation process involved four experienced annotators who underwent multiple pilot rounds to refine the taxonomy and annotation guide. The goal was to capture not just explicit hate but also implicit meanings and cultural references within the visual content.

Datasets

The paper introduces the VIDA (Visual Incel Data Archive) dataset.

  • Source: Collected from Italian and Anglophone Incel forums.
  • Content: Images and memes.
  • Size: The full dataset contains 445,442 images and memes. A subset of 2181 images was manually annotated for this study, along with their discursive textual context.
  • Novelty: This is a multimodal, theory-oriented dataset specifically designed for analyzing visual extremism within the Incelosphere, addressing a gap in existing resources.

Key Insights and Ideas

  • Visuals are Crucial for Extremism: The study highlights that images and memes are not just supplementary but are central to the spread of extremist ideologies, carrying complex meanings and cultural nuances.
  • Implicit Hate is Pervasive: Explicit hate speech is less common than stereotypical content, body shaming, and objectification, which are often conveyed implicitly through visuals and accompanying text.
  • Community-Specific Nuances: The research suggests differences in how political ideologies and cultural references are framed between Italian and English-speaking Incel communities, indicating the importance of localized analysis.
  • Meme Culture as a Vehicle for Hate: The paper emphasizes how the unique expressive forms, lexicon, and inside jokes within online communities like the Incelosphere are leveraged through visual culture to recruit members and disseminate hateful ideas.
  • Beyond Simple Image Recognition: Detecting hate in visual content requires understanding its form, content, and stance, as well as its interplay with textual context, moving beyond basic image classification.

Key Results

  • The annotated dataset of 2181 images revealed a prevalence of stereotypical content targeting both men and women, as well as ethnic and racial stereotypes.
  • Body shaming and moral shaming were identified as common forms of discrimination and abuse embedded in the visual content.
  • While explicit misogyny was rare, images were often found to be extremely explicit when misogynistic content was present.
  • The study achieved significant inter-annotator agreement on the developed taxonomy, validating its robustness for analyzing this type of content.
  • Quantitative analysis showed that simple images, rather than complex memes, often carried the most offensive messages, underscoring the need for multimodal analysis.

Limitations and Future Work

  • Generalizability of Annotation Protocol: The specific classes developed for this dataset might make it difficult to apply the annotation protocol directly to other data sources without adaptation.
  • Focus on Explicit Hate: The annotators tended to limit inferences about hateful content to what was explicitly expressed in the image and associated text, potentially overlooking deeper contextual or backgrounded information.
  • Future Work:
    • Release the full, anonymized dataset (including threads, posts, and more images).
  • Apply advanced anonymization techniques like Differential Privacy.
  • Evaluate the taxonomy's applicability to other domains and potentially generalize categories.
  • Train classifiers on the dataset and employ human-in-the-loop paradigms to scale annotations.
  • Further investigate the links between visual references in Incel communities and other subcultures (e.g., otaku geek subcultures).

Summary updated

BibTeX
@inproceedings{anastasi2024vida,
  author = {Selenia Anastasi and Florian Schneider and Chris Biemann and Tim Fischer},
  title = {{VIDA: The Visual Incel Data Archive. A Theory-oriented Annotated Dataset To Enhance Hate Detection Through Visual Culture}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
}
Figure