NLP_goats@DravidianLangTech 2025: Towards Safer Social Media: Detecting Abusive Language Directed at Women in Dravidian Languages

Vijay Karthick Vaidyanathan, Srihari V K, Thenmozhi DurairajNAACL 2025

TL;DR

This paper tackles the critical challenge of detecting abusive language directed at women on social media, specifically in Tamil and Malayalam. By fine-tuning multilingual transformer models like mBERT and XLM-R on a custom YouTube comments dataset, the researchers achieved promising results in identifying and filtering harmful content, paving the way for safer online spaces.

What Problem Does This Paper Address?

In today's interconnected world, social media platforms are essential for communication, but they've also become breeding grounds for online abuse, particularly targeting women. This gender-based abuse, characterized by degrading, harassing, and silencing language, reflects and exacerbates societal inequalities. The psychological and social impact on victims can be devastating, leading many to abandon these platforms. Identifying and eradicating such abusive content is therefore crucial for fostering safe digital environments and ensuring equal opportunities.

While abusive language detection has been explored for high-resource languages like English, it presents significant challenges for low-resource languages such as Tamil and Malayalam. These languages have complex grammars, distinctive scripts, and often hide abusive patterns within implicit biases, sarcasm, stereotypes, and idiomatic expressions. Furthermore, the scarcity of annotated datasets and linguistic tools for these languages compounds the difficulty. This research aims to bridge this gap by developing automated systems for detecting abusive content specifically targeting women in Tamil and Malayalam.

Methods

The study leverages the power of pre-trained multilingual transformer models, specifically mBERT (Multilingual BERT) and XLM-R (Cross-lingual Language Model - RoBERTa). These models are chosen for their ability to capture long-range dependencies in text through self-attention mechanisms, which is vital for understanding the nuanced and contextual nature of abusive language.

The core methodology involves fine-tuning these pre-trained models on a custom dataset of YouTube comments in Tamil and Malayalam. The process includes:

  • Data Preprocessing: This crucial step involves normalizing Unicode characters, converting text to lowercase, and removing unnecessary punctuation, emojis, and numbers. This noise reduction aims to improve the model's ability to classify abusive comments accurately.
  • Label Encoding: Dataset labels were encoded numerically ("Non-Abusive" as 0, "Abusive" as 1) for compatibility with machine learning models.
  • Fine-tuning: The pre-trained mBERT and XLM-R models were then fine-tuned on the preprocessed dataset. The output layer of each model was modified to perform binary classification (Abusive vs. Non-Abusive).

The researchers highlight that these transformer models, with their bidirectional self-attention, are particularly well-suited for understanding contextual subtleties and are effective even with limited annotated data, making them ideal for low-resource language scenarios.

Datasets

The primary dataset used in this study consists of YouTube comments in Tamil and Malayalam. These comments were collected specifically concerning sensitive issues, making them relevant for identifying abusive language directed at women. The paper does not explicitly state the size of the dataset but provides label distributions for both languages.

  • Malayalam Abusive Language Dataset:
    • Abusive: 1531 instances
    • Non-Abusive: 1402 instances
  • Tamil Abusive Language Dataset:
    • Abusive: 1658 instances
    • Non-Abusive: 1598 instances

The paper mentions that this dataset was provided by previous works on abusive language detection in Dravidian languages, indicating a reliance on existing benchmarks within this specific research community.

Key Insights and Ideas

  • Multilingual Transformers Shine in Low-Resource Settings: The study demonstrates that powerful pre-trained multilingual models like mBERT and XLM-R can be effectively adapted for abusive language detection in low-resource languages like Tamil and Malayalam, even with limited annotated data.
  • Context is King (and Difficult): The inherent contextual nature of abusive language, including sarcasm, implicit bias, and veiled speech, poses a significant challenge. Models struggle to accurately differentiate emotionally charged but non-abusive comments from genuinely abusive ones.
  • Code-Mixing is a Hurdle: The models encountered difficulties with code-mixed (mixing of languages) and transliterated text, which is common in social media. This suggests a need for more robust handling of such linguistic phenomena.
  • The Importance of Fine-Tuning: While pre-training provides a strong foundation, fine-tuning on domain-specific data (YouTube comments in this case) is crucial for achieving good performance in the target task.

Key Results

The study evaluated the performance of mBERT and XLM-R using F1-scores.

  • mBERT:
    • Tamil: Achieved an F1-score of 0.75.
  • Malayalam: Achieved an F1-score of 0.68.
  • XLM-R:
    • Tamil: Achieved an F1-score of 0.68.
  • Malayalam: Achieved an F1-score of 0.65.

Overall, mBERT outperformed XLM-R in detecting abusive content in both Tamil and Malayalam. The paper notes that mBERT was particularly effective, but struggled with accurately classifying non-abusive Malayalam content. XLM-R also showed good performance but fell slightly short of mBERT.

Limitations and Future Work

The authors acknowledge several limitations:

  • Dataset Bias: The dataset, primarily sourced from YouTube comments, might not fully represent abusive patterns across all social media platforms, potentially leading to bias.
  • Contextual Understanding Gaps: The models still struggle with sarcasm, implicit biases, and slang, leading to occasional false positives and negatives.
  • Limited Training Data: The relatively small amount of annotated data for Tamil and Malayalam impacts the overall results.
  • Computational Load: mBERT and XLM-R are computationally intensive, posing challenges for real-time deployment on resource-constrained devices.

Future work suggested includes:

  • Increasing Dataset Sizes: Expanding the annotated datasets for both languages.
  • Incorporating Additional Knowledge: Exploring ways to integrate external knowledge or linguistic resources.
  • Refining Techniques: Developing improved feature engineering and fine-tuning strategies to enhance accuracy and fairness.
  • Addressing Code-Mixing: Developing more robust methods for handling code-mixed and transliterated text.
  • Error Mitigation: Focusing on strategies to reduce false positives, particularly in differentiating emotionally charged but non-abusive content.

Summary updated

BibTeX
@inproceedings{vaidyanathan2025nlp,
  author = {Vijay Karthick Vaidyanathan and Srihari V K and Thenmozhi Durairaj},
  title = {{NLP\_goats@DravidianLangTech 2025: Towards Safer Social Media: Detecting Abusive Language Directed at Women in Dravidian Languages}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure