Dll5143@DravidianLangTech 2025: Majority Voting-Based Framework for Misogyny Meme Detection in Tamil and Malayalam

Sarbajeet Pattanaik, Ashok Yadav, Vrijendra SinghNAACL 2025

TL;DR

This paper tackles the challenging task of detecting misogynistic memes in Tamil and Malayalam, languages often overlooked in NLP research. They propose a robust multimodal framework that combines predictions from three distinct models using a majority voting mechanism, achieving competitive results in a recent shared task.

What Problem Does This Paper Address?

The proliferation of social media has led to an increase in harmful online content, including misogynistic memes. While significant progress has been made in detecting such content in English, there's a pressing need to extend these efforts to low-resource languages, particularly those using Dravidian scripts like Tamil and Malayalam. These languages present unique challenges due to their linguistic diversity, cultural nuances, and the complex multimodal nature of memes (combining text and images). Accurately identifying misogyny in these contexts requires sophisticated models that can understand subtle expressions and cultural references, which are often missed by models trained on more widely resourced languages.

Methods

The core of the proposed solution is a majority voting-based framework that leverages the strengths of three different multimodal models. Each of these models takes a meme (image + text) as input and outputs a prediction of whether it's misogynistic or not. The final classification is determined by the consensus of these three models: if at least two out of three models agree on a classification, that classification is chosen.

The three individual models are:

  • M1: Combines a ResNet-50 (for image features) with Google's muril-large-cased (for text features).
  • M2: Uses OpenAI's CLIP ViT (for image features) and ai4bharat/indic-bert (for text features).
  • M3: Integrates a ResNet-50 (for image features) with ai4bharat/indic-bert (for text features).

Each model processes the image and text modalities separately, extracts features, and then fuses them for a final classification. The use of ai4bharat/indic-bert and muril-large-cased is particularly noteworthy as they are pre-trained on a wide range of Indian languages, making them suitable for Tamil and Malayalam.

Datasets

The paper utilizes datasets provided for the "Shared Task on Misogyny Meme Detection" organized as part of DravidianLangTech@NAACL 2025. These datasets were drawn from previous work by Ponnusamy et al. (2024). The paper provides statistics on the training, validation, and test splits for both Tamil and Malayalam, detailing the number of misogynistic and non-misogynistic memes in each.

Key Insights and Ideas

  • Majority Voting for Robustness: The simple yet effective majority voting mechanism significantly improves robustness. By aggregating predictions from diverse models, it mitigates individual model biases and enhances generalization, especially crucial for nuanced tasks like misogyny detection.
  • Multimodal Fusion is Key: The success of the framework highlights the necessity of combining both visual and textual information. Memes often rely on the interplay between image and text to convey their full meaning, and a single modality is insufficient.
  • Importance of Indic-Specific Models: The use of models like ai4bharat/indic-bert and muril-large-cased demonstrates their effectiveness in handling the complexities of Dravidian languages, including code-mixing and script-specific nuances.
  • Cultural Nuances Pose a Challenge: The error analysis reveals that understanding subtle cultural contexts, implicit biases, and gender stereotypes within Tamil and Malayalam memes remains a significant hurdle. Models can struggle with sarcasm, indirect references, and culturally specific humor.

Key Results

The proposed framework achieved competitive results in the DravidianLangTech@NAACL 2025 shared task:

  • Tamil: An F1 score of 0.77, ranking 5th overall.
  • Malayalam: An F1 score of 0.84, ranking 4th overall.

These scores indicate strong performance, especially considering the challenging nature of the task and the low-resource languages involved. The paper also presents detailed performance breakdowns for each individual model (M1, M2, M3) on both Tamil and Malayalam datasets, showing how the majority voting approach generally outperforms individual models.

Limitations and Future Work

The authors acknowledge several limitations:

  • Class Imbalance: The Tamil dataset, in particular, exhibits a significant class imbalance (only 25% misogynistic memes), which can bias models towards the majority class.
  • Cultural Nuances and Implicit Misogyny: Detecting subtle, implicit misogyny, especially those related to cultural stereotypes or body-part objectification, remains challenging.
  • Code-Mixing and Script-Specific Features: The complexity of code-mixed language (e.g., Tamil-English) and the need for script-specific features in Malayalam pose ongoing difficulties.

Future work suggested includes:

  • Developing models that can better distinguish satire from hateful content, particularly in body-part-related contexts.
  • Exploring specialized pre-training or script-specific models to better handle the linguistic intricacies of Dravidian languages.
  • Integrating culturally specific and contextually rich datasets.
  • Investigating advanced techniques like weighted ensemble methods and fine-grained contextual embeddings.

Summary updated

BibTeX
@inproceedings{pattanaik2025dll,
  author = {Sarbajeet Pattanaik and Ashok Yadav and Vrijendra Singh},
  title = {{Dll5143@DravidianLangTech 2025: Majority Voting-Based Framework for Misogyny Meme Detection in Tamil and Malayalam}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure