InnovationEngineers@DravidianLangTech 2025: Enhanced CNN Models for Detecting Misogyny in Tamil Memes Using Image and Text Classification

Kogilavani Shanmugavadivel, Malliga Subramanian, Pooja Sree M, Palanimurugan V, Roshini Priya KNAACL 2025

TL;DR

This paper tackles the challenge of detecting misogynistic memes in Tamil, a low-resource language, by developing a multimodal deep learning approach that combines image and text analysis. Their Vision-Language Model (VLM) achieved the highest accuracy, demonstrating the power of integrating visual and textual cues for this complex task.

What Problem Does This Paper Address?

The proliferation of harmful content on social media, particularly misogynistic memes, poses a significant threat to civil discourse. Detecting such content is especially challenging in low-resource languages like Tamil, where existing NLP tools and datasets are scarce. This paper aims to bridge this gap by creating a binary classification model to identify misogynistic memes in Tamil, addressing the need for effective tools to combat online abuse in underrepresented linguistic communities.

Methods

The researchers propose a multimodal deep learning architecture that leverages multiple powerful models to analyze both the visual and textual components of memes. Their approach integrates:

  • EfficientNet: Used for extracting rich image features.
  • BERT (Bidirectional Encoder Representations from Transformers): Employed to generate language-specific text embeddings, capturing nuanced semantic meaning.
  • LSTM (Long Short-Term Memory): Utilized to process sequential text patterns, understanding the flow and context of the text.
  • Vision-Language Models (VLM): This is a key component, designed to comprehend the intricate relationship between images and text, a crucial aspect of meme understanding.

The core idea is to fuse these extracted features to build a robust classification model.

Datasets

The study utilized a curated dataset consisting of 1,776 Tamil memes. This dataset was split into training (1,136 memes), validation (284 memes), and testing (356 memes) sets. Each entry included an image ID, a label (misogynistic or non-misogynistic), and the corresponding text transcription. The paper does not explicitly state if this dataset is newly introduced or a modification of an existing one, but it was prepared for the Dravidian-LangTech 2025 Shared Task.

Key Insights and Ideas

  • Multimodal is King: The most significant insight is that combining visual and textual information is crucial for accurately detecting misogyny in memes. Single-modality approaches (like text-only or image-only) are insufficient.
  • VLM's Superiority: Vision-Language Models (VLMs) demonstrated a remarkable ability to understand the interplay between image and text, leading to the highest accuracy. This highlights the importance of models specifically designed to bridge the gap between visual and linguistic modalities.
  • Low-Resource Language Focus: The paper underscores the difficulty and importance of developing such tools for low-resource languages, where data scarcity and cultural nuances present unique challenges.
  • Balancing Precision and Recall: While VLM achieved the highest accuracy, EfficientNet showed a better balance between precision and recall (higher F1-score), indicating its strength in correctly identifying both positive and negative cases without excessive false positives or negatives.

Key Results

The study compared the performance of several models:

  • VLM: Achieved the highest accuracy at 85.0% and an F1-score of 70.8.
  • EfficientNet: Showed strong performance with 82.0% accuracy and the highest F1-score of 72.5, indicating a good balance between precision and recall.
  • BERT: Achieved 79.5% accuracy and a 65.0 F1-score, performing well on text but lagging behind multimodal approaches.
  • LSTM: Recorded 75.0% accuracy and a 67.5 F1-score, proving useful for sequential text but less effective than VLMs.

Notably, their proposed approach ranked 12th out of 118 participants in the Tamil language track of the Dravidian-LangTech 2025 competition, showcasing its competitive performance.

Limitations and Future Work

Limitations:

  • Class Imbalance: The paper acknowledges that models like BERT and LSTM struggle with class imbalance, which can affect performance.
  • Accuracy vs. F1-Score: While VLM achieved the highest accuracy, EfficientNet's superior F1-score suggests that further optimization might be needed for VLM to achieve a better balance between precision and recall.

Future Work:

  • Improved Feature Fusion: The authors suggest exploring enhanced feature fusion techniques to further boost classification accuracy.
  • Addressing Class Imbalance: Techniques like oversampling, class weighting, domain-specific models, hyperparameter tuning, and data augmentation are mentioned as potential improvements for models struggling with imbalanced datasets.
  • Fine-tuning and Data Augmentation: These are suggested as ways to enhance the performance of models like EfficientNet.

Summary updated

BibTeX
@inproceedings{shanmugavadivel2025innovationengineers,
  author = {Kogilavani Shanmugavadivel and Malliga Subramanian and Pooja Sree M and Palanimurugan V and Roshini Priya K},
  title = {{InnovationEngineers@DravidianLangTech 2025: Enhanced CNN Models for Detecting Misogyny in Tamil Memes Using Image and Text Classification}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure