RMKMavericks@DravidianLangTech 2025: Tackling Abusive Tamil and Malayalam Text Targeting Women: A Linguistic Approach

Sandra Johnson, Boomika E, Lahari PNAACL 2025

TL;DR

This paper tackles the critical issue of online abuse targeting women in Tamil and Malayalam, languages often underserved by existing NLP tools. The authors developed and evaluated machine learning models (SVM for Tamil, MNB for Malayalam) using TF-IDF for feature extraction, achieving promising F1 scores and highlighting the importance of language-specific approaches for effective abusive content detection.

What Problem Does This Paper Address?

The pervasive nature of social media has unfortunately led to a surge in online abuse, with women and vulnerable groups being disproportionately targeted. This problem is particularly acute in regional languages like Tamil and Malayalam, where the linguistic diversity and complexity, coupled with a scarcity of specialized tools, make automated detection of abusive content a significant challenge. Existing NLP models often struggle with code-mixed languages (where multiple languages are blended) and the nuances of informal online communication. This paper aims to bridge this gap by developing and evaluating machine learning models specifically tailored for identifying abusive text in Tamil and Malayalam, thereby contributing to safer online environments for speakers of these languages.

Methods

The researchers employed a standard machine learning pipeline, focusing on preprocessing, feature extraction, and model selection.

  • Preprocessing: This involved cleaning and normalizing text data, including converting to lowercase, removing special characters and punctuation, and tokenization. For code-mixed data, specific steps were taken to preserve linguistic structure. Stopwords were selectively removed, and whitespace inconsistencies were corrected.
  • Feature Extraction: Two common techniques were used:
  • Bag of Words (BOW): This method represents text as a vector of word frequencies, capturing how often words appear in a document.
  • TF-IDF (Term Frequency-Inverse Document Frequency): This technique weighs words based on their importance in the dataset, emphasizing less common but more significant terms.
  • Model Selection:
    • For Tamil, a Support Vector Machine (SVM) was chosen. SVMs are known for their ability to handle high-dimensional and complex data, making them suitable for intricate text classification tasks.
  • For Malayalam, a Multinomial Naive Bayes (MNB) model was selected. MNB is computationally efficient and performs well with categorical data, making it a good choice for this task.
  • Training and Optimization: The models were trained on provided datasets. For the Tamil SVM, hyperparameter tuning was performed using a grid search to optimize parameters like regularization (C) and kernel types. The MNB model for Malayalam was trained directly without extensive hyperparameter tuning due to its inherent simplicity.

Datasets

The study utilized datasets provided by the organizers for both Tamil and Malayalam, which were split into training and testing sets. Additionally, an external dataset was incorporated for Tamil to enhance model diversity and performance. The paper does not explicitly name or describe the creation of these datasets, but they are referred to as "code-mixed" and "language-specific."

Key Insights and Ideas

  • Language-Specific Models are Crucial: The success of using SVM for Tamil and MNB for Malayalam underscores the importance of tailoring models to the specific linguistic characteristics and complexities of each language. A one-size-fits-all approach is unlikely to be effective.
  • TF-IDF Remains a Viable Baseline: Despite the rise of more complex deep learning models, TF-IDF, when combined with appropriate algorithms, can still yield competitive results for text classification tasks, especially in resource-constrained scenarios.
  • Code-Mixed Data Presents Unique Challenges: The paper implicitly highlights the ongoing difficulty in handling code-mixed languages, where the blending of multiple languages requires careful preprocessing to maintain linguistic integrity.

Key Results

The performance of the models was evaluated using standard metrics: Accuracy, Precision, Recall, F1-score, and AUC.

  • Tamil: Achieved an F1-score of 0.6196.
  • Malayalam: Achieved an F1-score of 0.6484.

Confusion matrices were also generated, showing the distribution of true positives, true negatives, false positives, and false negatives for both languages. The AUC-ROC curves indicated good model performance, with AUC values of 0.94 for Tamil and 0.89 for Malayalam. Precision-Recall curves also demonstrated reasonable performance in detecting positive (abusive) examples.

Limitations and Future Work

The authors acknowledge several limitations:

  • Informal and Unstructured Data: The highly informal and unstructured nature of code-mixed Dravidian languages, with their intricate linguistic variances, poses a significant challenge.
  • Small Dataset Size: The limited size of the datasets, particularly for Malayalam, may have impacted the generalizability of the models.
  • Limited Feature Extraction: Relying solely on TF-IDF and BOW representations might have missed crucial contextual and semantic links between words.
  • Exclusion of Advanced Models: Due to computational limitations, advanced deep learning models like transformer-based architectures were not explored.

Future work suggested includes:

  • Contextual Embeddings: Incorporating contextual embeddings to capture richer semantic information.
  • Dataset Expansion: Broadening the datasets to improve model generalizability.
  • Multimodal Approaches: Exploring the combination of textual data with audio and visual information for a more comprehensive analysis of abusive language.
  • Investigating Transformer Models: Exploring the potential of advanced models like BERT and RoBERTa for improved performance.

Summary updated

BibTeX
@inproceedings{johnson2025rmkmavericks,
  author = {Sandra Johnson and Boomika E and Lahari P},
  title = {{RMKMavericks@DravidianLangTech 2025: Tackling Abusive Tamil and Malayalam Text Targeting Women: A Linguistic Approach}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure