JAS@DravidianLangTech 2025: Abusive Tamil Text targeting Women on Social Media

B Saathvik, Janeshvar Sivakumar, Thenmozhi DurairajNAACL 2025

TL;DR

This paper tackles the critical issue of detecting abusive comments targeting women on Tamil social media. By fine-tuning Google's multilingual MuRIL transformer model on a custom dataset, the researchers achieved promising results, demonstrating the potential of advanced NLP models for this under-resourced language.

What Problem Does This Paper Address?

Social media has become a breeding ground for abusive language, with women disproportionately targeted. This issue is particularly challenging in low-resource languages like Tamil, where readily available, high-quality datasets and NLP tools are scarce. The paper aims to bridge this gap by developing an automated system to identify abusive comments directed at women on Tamil social media platforms, thereby contributing to a safer online environment.

Methods

The core of this work lies in leveraging a powerful pre-trained transformer model, Google's MuRIL (Multilingual Representations for Indian Languages). MuRIL, trained on 17 Indian languages, is fine-tuned for the specific task of binary classification: abusive vs. non-abusive comments. The authors employed a standard preprocessing pipeline, including:

  • Removal of irrelevant elements: Numbers, hyperlinks, email addresses, and emojis were removed as they don't contribute to sentiment analysis.
  • Character normalization: English characters were converted to lowercase while preserving the Tamil script's integrity.
  • Special character and punctuation removal: This step aimed to standardize text and reduce noise.
  • Stop word removal: Common words that don't carry significant semantic meaning were removed to reduce data dimensionality and improve processing efficiency.

The preprocessed data was then tokenized using MuRIL's tokenizer, and sequences were padded and truncated to a uniform length. The fine-tuning process involved training the model for 5 epochs with a batch size of 16, utilizing the Hugging Face Trainer API. The best model was selected based on the highest F1-score on the validation set.

For comparison, the authors also evaluated traditional machine learning models like Logistic Regression, SVM, LinearSVC, and XGBoost on TF-IDF transformed data.

Datasets

The primary dataset used in this study is a collection of 2790 manually annotated YouTube comments in Tamil, sourced from Priyadharshini et al. (2022, 2023). This dataset is divided into:

  • 1424 "Non-Abusive" comments
  • 1366 "Abusive" comments

The comments are noted to contain a mix of letters, numbers, symbols, special characters, emojis, emails, and hyperlinks, making them representative of real-world social media content. The paper does not introduce a new benchmark dataset but utilizes an existing one for the task.

Key Insights and Ideas

  • Transformer models are powerful for low-resource languages: The success of MuRIL, a multilingual model, highlights the effectiveness of pre-trained transformers in handling languages with limited resources, overcoming the need for extensive language-specific feature engineering.
  • Data quality is paramount: The authors acknowledge that the lack of large, high-quality datasets is a significant hurdle for low-resource languages, emphasizing that improving dataset quality is crucial for better performance.
  • Contextual understanding is key: MuRIL's ability to capture contextual relationships between words, thanks to its self-attention mechanism, is vital for accurately classifying nuanced and potentially offensive language.

Key Results

The fine-tuned MuRIL model achieved:

  • Accuracy: 77.76%
  • F1-score: 77.65%

In the context of the broader task, the model ranked 10th with an F1-score of 0.7687 on the test dataset. While MuRIL significantly outperformed traditional models like Logistic Regression, SVM, LinearSVC, and XGBoost (which achieved F1-scores ranging from 0.65 to 0.70), there is still room for improvement.

Limitations and Future Work

The authors acknowledge several limitations:

  • Dataset biases: The performance of the model is influenced by biases present in the dataset, which can limit its generalization to diverse scenarios.
  • Architectural and loss function limitations: Current architectural choices and loss functions might not fully capture the complexities of real-world abusive language.
  • Low generalizability: The small size of the dataset poses a challenge for achieving high generalizability to real-world scenarios.
  • Ethical concerns: Potential biases in AI decisions require continuous monitoring.
  • Computational demands: High computational requirements present scalability challenges.

Future work suggested includes exploring advanced data augmentation techniques and further fine-tuning to enhance performance, ultimately contributing to better automated content moderation for underrepresented languages.

Summary updated

BibTeX
@inproceedings{saathvik2025jas,
  author = {B Saathvik and Janeshvar Sivakumar and Thenmozhi Durairaj},
  title = {{JAS@DravidianLangTech 2025: Abusive Tamil Text targeting Women on Social Media}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure