LexiLogic@DravidianLangTech 2025: Detecting Misogynistic Memes and Abusive Tamil and Malayalam Text Targeting Women on Social Media

Niranjan Kumar M, Pranav Gupta, Billodal Roy, Souvik BhattacharyyaNAACL 2025

TL;DR

This paper tackles the critical issue of detecting misogynistic memes and abusive text targeting women on social media, specifically in low-resource languages like Tamil and Malayalam. By fine-tuning specialized pre-trained language models and employing data augmentation techniques, the authors achieve significant improvements in accuracy, paving the way for safer online spaces.

What Problem Does This Paper Address?

Social media has become a double-edged sword: a powerful tool for connection and expression, but also a breeding ground for harmful content. Misogyny and abusive language targeting women are particularly pervasive and damaging, impacting mental health and reinforcing societal biases. While significant progress has been made in detecting such content in high-resource languages like English, underrepresented languages like Tamil and Malayalam remain largely underserved. This gap is due to a lack of specialized datasets and the inherent linguistic and cultural complexities of these languages, which general-purpose NLP models often fail to capture. This paper aims to bridge this gap by developing effective detection systems for Tamil and Malayalam.

Methods

The core of this work lies in leveraging and adapting advanced NLP techniques for low-resource languages. The authors utilize pre-trained language models specifically developed for Tamil and Malayalam: L3Cube-Tamil-BERT and L3Cube-Malayalam-BERT. These models are then fine-tuned on curated datasets for two distinct tasks: misogynistic meme detection and abusive text detection targeting women.

To further enhance performance, the researchers employed several data augmentation strategies. These include:

  • Synonym Replacement: Swapping words with their synonyms to increase vocabulary diversity.
  • Back-Translation: Translating text to another language and then back to the original, creating paraphrased versions.
  • Contextual Data Augmentation: Generating new data points that maintain the original context but introduce variations.

These techniques help to enrich the training data, improve class balance, and enable the models to better generalize to unseen examples.

Datasets

The paper focuses on tasks from the Dravidian Language Technology Workshop, specifically:

  • Offensive Language Identification (Competition 20701): This task involved classifying social media text in Tamil and Malayalam as either offensive or non-offensive.
  • Meme Classification for Tamil and Malayalam (Competition 20856): This task focused on classifying memes into categories such as offensive, humorous, or neutral, using textual data extracted from the memes.

The authors also mention the IndicXNLI dataset, which is a benchmark for Natural Language Inference (NLI) across 11 Indian languages, including Tamil and Malayalam. While IndicXNLI is used for evaluating NLI performance and understanding linguistic diversity, the primary datasets for this work are the competition-specific datasets for offensive language and meme classification.

Key Insights and Ideas

  • Language-Specific Models Trump General Models: The study strongly suggests that pre-trained models tailored to specific low-resource languages (like L3Cube-Tamil-BERT and L3Cube-Malayalam-BERT) significantly outperform general-purpose multilingual models (like mBERT and XLM-R) for tasks involving those languages. This highlights the importance of capturing unique linguistic nuances.
  • Data Augmentation is Crucial: The effectiveness of data augmentation techniques in improving detection accuracy and contextual understanding for nuanced abusive content is a key takeaway. This is particularly vital for low-resource languages where annotated data is scarce.
  • Cultural Nuances Matter: The success of the specialized models underscores the need to account for cultural context and linguistic subtleties when detecting harmful content, especially when targeting specific demographics like women.

Key Results

The paper reports impressive results on the competition tasks:

  • Offensive Language Identification:
    • L3Cube-Tamil-BERT achieved a macro F1 score of 78.24.
  • L3Cube-Malayalam-BERT achieved a macro F1 score of 70.01. These scores placed the models among the top performers for this task.

  • Meme Classification:

    • L3Cube-Tamil-BERT achieved a macro F1 score of 73.16.
  • L3Cube-Malayalam-BERT achieved a macro F1 score of 82.94. The Malayalam model, in particular, achieved a high score, ranking 3rd on the leaderboard.

The paper also notes that for the misogyny task, they obtained macro F1 scores of 83.58/82.94 on L3Cube-Bert-Malayalam and 73.16/73.8 on L3Cube-Bert-Tamil, placing 9th in both.

Limitations and Future Work

The authors acknowledge several limitations:

  • Static Datasets: The models are trained on static datasets, which may not reflect the evolving nature of online abuse and misogyny. Changing trends can render detectors outdated.
  • Bias: There's a risk that these models might inadvertently discriminate against certain social media users.
  • Adversarial Exploitation: Malicious actors could potentially exploit the limitations of these models to circumvent detection and continue posting undesirable content.

To address these, future work should focus on:

  • Active Learning and Continual Learning: Implementing these techniques to ensure detectors remain up-to-date with evolving trends and maintain a balanced representation of new and existing content.
  • Larger Datasets: Exploring larger unsupervised and supervised datasets to further improve performance metrics, especially in low-resource settings.
  • Hybrid Approaches: Investigating hybrid methods or advanced fine-tuning strategies to further enhance model performance.

Summary updated

BibTeX
@inproceedings{m2025lexilogic,
  author = {Niranjan Kumar M and Pranav Gupta and Billodal Roy and Souvik Bhattacharyya},
  title = {{LexiLogic@DravidianLangTech 2025: Detecting Misogynistic Memes and Abusive Tamil and Malayalam Text Targeting Women on Social Media}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure