KEC_AI_VSS_run2@DravidianLangTech 2025: Abusive Tamil and Malayalam Text targeting Women on Social Media

Kogilavani Shanmugavadivel, Malliga Subramanian, Sathiyaseelan S, Suresh Babu K, Vasikaran SNAACL 2025

TL;DR

This paper tackles the critical challenge of detecting gender-based abusive content targeting women on social media in low-resource languages like Tamil and Malayalam. By curating annotated datasets and applying traditional machine learning models, the authors achieved impressive results, with SVM excelling for Tamil and Random Forest for Malayalam, highlighting the need for language-specific approaches.

What Problem Does This Paper Address?

Social media has become a double-edged sword, facilitating communication while also becoming a breeding ground for serious issues like gender-based abuse. This abuse, often derogatory and threatening, carries significant psychological, social, and professional consequences for victims. The challenge is amplified in low-resource languages like Tamil and Malayalam, where robust automated content moderation systems are scarce due to a lack of annotated data and the inherent complexity of linguistic nuances, including implicit biases and coded expressions. This paper aims to bridge this gap by developing effective models for detecting gender-based abuse in Tamil and Malayalam YouTube comments, ultimately contributing to safer online spaces for women.

Methods

The researchers employed a standard machine learning pipeline for text classification. This involved:

  • Data Preprocessing: Cleaning raw YouTube comments by removing special characters, emojis, punctuation, and URLs. This was followed by tokenization, stop-word removal, and text normalization to standardize spelling and slang. To address class imbalance in the datasets, techniques like over-sampling and under-sampling were applied.
  • Feature Extraction: Two main approaches were used:
  • TF-IDF (Term Frequency-Inverse Document Frequency): This method weighs words based on their frequency within comments, down-weighting common terms.
  • Word Embeddings: Techniques like Word2Vec, FastText, and BERT were utilized to capture contextual meaning and understand relationships between words, which is crucial for distinguishing subtle forms of abuse.
  • Model Development: Several machine learning algorithms were trained and fine-tuned for classification:
  • Random Forest
    • Support Vector Machine (SVM)
    • K-Nearest Neighbor (KNN)
    • Gradient Boosting
    • AdaBoost

The dataset was split into training, validation, and test sets for effective generalization.

Datasets

The study utilized two primary datasets:

  • Tamil Dataset: Consisted of 2,790 records, evenly split between abusive (1,366) and non-abusive (1,402) comments. This balanced distribution is ideal for unbiased model training.
  • Malayalam Dataset: Comprised 2,933 records, with a slight imbalance: 1,531 abusive and 1,402 non-abusive comments. The authors note this imbalance reflects real-world scenarios where abusive language might be more prevalent in certain contexts and adds to the challenge due to the language's complexity.

The paper mentions participation in a shared task on abusive comment detection, implying the use of datasets associated with that task.

Key Insights and Ideas

  • Language-Specific Models are Crucial: The significant difference in performance between Tamil and Malayalam models (SVM for Tamil, Random Forest for Malayalam) strongly suggests that a one-size-fits-all approach is insufficient for abusive content detection in diverse languages. Tailored models are essential.
  • Traditional ML Still Holds Its Ground: Despite the rise of deep learning, traditional ML models like SVM and Random Forest, when combined with effective feature engineering (TF-IDF and word embeddings), can achieve high performance, especially in low-resource settings where large deep learning models might be data-hungry.
  • Implicit Abuse is a Major Hurdle: The paper highlights that models need to capture not just explicit abusive language but also implicit bias and coded expressions, which are particularly challenging for automated systems.
  • Dataset Balance Matters (and Real-World Imbalance is a Challenge): While a balanced dataset is ideal for training, the Malayalam dataset's natural imbalance presents a more realistic, albeit harder, scenario that requires careful handling.

Key Results

  • Tamil: The Support Vector Machine (SVM) model achieved the highest micro F1 score of 0.95, demonstrating exceptional performance in classifying abusive comments.
  • Malayalam: The Random Forest model performed best, achieving a micro F1 score of 0.72. While lower than the Tamil result, this is considered robust given the language's complexity and dataset imbalance.
  • Shared Task Performance: The system ranked 13th for Malayalam and 34th for Tamil out of 160 teams in a shared task on abusive comment detection, indicating competitive performance.

Limitations and Future Work

Limitations:

  • Dataset Size: The limited size of the datasets may affect the generalizability of the models to broader contexts.
  • Malayalam Dataset Imbalance: The imbalance in the Malayalam dataset could lead to biased predictions.
  • Reliance on Traditional Models: The use of traditional machine learning models might not fully capture the complex linguistic patterns of abusive language compared to deep learning techniques.
  • Absence of Deep Learning: The paper acknowledges that the absence of deep learning techniques limits the potential for achieving even higher accuracy.

Future Work:

The authors suggest that deep learning models and contextualized embeddings would be highly beneficial for improving performance. Future work will focus on enlarging datasets and fine-tuning classification techniques to enhance accuracy and generalizability for real-world applications. The code for this shared task is available on Github.

Summary updated

BibTeX
@inproceedings{shanmugavadivel2025kec,
  author = {Kogilavani Shanmugavadivel and Malliga Subramanian and Sathiyaseelan S and Suresh Babu K and Vasikaran S},
  title = {{KEC\_AI\_VSS\_run2@DravidianLangTech 2025: Abusive Tamil and Malayalam Text targeting Women on Social Media}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure