KEC_TECH_TITANS@DravidianLangTech 2025: Abusive Text Detection in Tamil and Malayalam Social Media Comments Using Machine Learning

Malliga Subramanian, Kogilavani Shanmugavadivel, Deepiga P, Dharshini S, Ananthakumar S, Praveenkumar CNAACL 2025

TL;DR

This paper tackles the crucial problem of detecting abusive language in Tamil and Malayalam social media comments, a significant challenge due to the low-resource nature of these languages and the prevalence of code-mixing. The authors found that FastText and CNN models achieved the best performance, highlighting their effectiveness for this complex task.

What Problem Does This Paper Address?

The proliferation of social media has unfortunately led to a surge in online abuse, with women often being disproportionately targeted. While significant research exists for abusive language detection in English, low-resource languages like Tamil and Malayalam remain largely under-explored. A major hurdle in these languages is "code-mixing," where users seamlessly blend words and phrases from both languages within a single comment. This linguistic complexity, coupled with informal spelling and contextual nuances, makes it difficult for traditional natural language processing (NLP) models to accurately identify abusive content. This paper aims to bridge this gap by applying and evaluating various machine learning models for abusive text detection in code-mixed Tamil and Malayalam social media comments.

Methods

The researchers employed a range of machine learning models, starting with established deep learning architectures and progressing to more advanced techniques. The models evaluated include:

  • Recurrent Neural Networks (RNNs): Specifically, GRU (Gated Recurrent Unit) and LSTM (Long Short-Term Memory) networks, known for their ability to capture sequential dependencies in text. Bidirectional LSTMs were also used to process sequences in both forward and backward directions, enhancing contextual understanding.
  • Convolutional Neural Networks (CNNs): These models are effective at extracting local features, akin to n-grams, making them computationally efficient for classification.
  • FastText: An embedding-based model that leverages pre-trained word vectors and averages them for classification.
  • XGBoost: A gradient boosting framework that uses decision trees, typically strong for structured data classification.

Before feeding the data into these models, standard preprocessing steps were applied, including text cleaning (removing punctuation, special characters, and emojis, with emojis converted to textual descriptions) and tokenization (splitting text into individual words or sub-word units). Feature extraction was performed using TF-IDF (Term Frequency-Inverse Document Frequency) to weigh the importance of words.

Datasets

The study utilized a newly curated dataset of 5,000 code-mixed Tamil and Malayalam social media comments collected from YouTube. Each comment was manually annotated with binary labels: "Abusive" or "Non-Abusive." The average sentence length in the dataset is one.

Key Insights and Ideas

  • Code-mixing is a significant challenge: The inherent variations in spelling, grammar, and transliteration within code-mixed text pose a substantial obstacle for existing NLP models.
  • FastText and CNNs excel: Despite the complexity, models like FastText and CNNs demonstrated superior performance, suggesting their robustness in handling the nuances of code-mixed languages for this task.
  • Traditional models struggle with long dependencies: GRU and LSTM models, while capable of handling sequential data, showed limitations in dealing with longer text dependencies, leading to more errors.
  • Class imbalance matters: The dataset exhibited class imbalance, which negatively impacted the F1-score, biasing predictions towards the majority class.

Key Results

The study evaluated models using Accuracy, Precision, Recall, and F1-Score. The FastText and CNN models achieved the highest F1-Score of 0.73, outperforming other evaluated models. GRU and LSTM models achieved an F1-Score of 0.69, while Bidirectional LSTM and XGBoost performed lower with F1-Scores of 0.34 and 0.67 respectively. Qualitatively, the CNN and FastText models, while performing well, occasionally misclassified non-abusive comments as abusive. GRU and LSTM models struggled more with longer comments.

Limitations and Future Work

The authors acknowledge several limitations:

  • Data Imbalance: The uneven distribution of abusive vs. non-abusive comments can lead to biased predictions.
  • Code-Mixed Challenges: The inherent complexities of code-mixed text remain a significant hurdle.
  • Generalization: Models trained on this specific dataset might not generalize well to other social media platforms or different language usage patterns.
  • Computational Constraints: Advanced transformer-based models, while promising, require substantial computational resources, making them difficult to deploy on low-end devices.
  • Contextual Limitations: Some comments require deeper contextual understanding that current models may not fully grasp.

Future work proposed includes exploring cutting-edge transformer-based architectures like BERT and RoBERTa, as well as investigating advanced feature representation techniques such as contextual embeddings and hybrid methods to better capture nuanced and context-dependent abusive language.

Summary updated

BibTeX
@inproceedings{subramanian2025kec,
  author = {Malliga Subramanian and Kogilavani Shanmugavadivel and Deepiga P and Dharshini S and Ananthakumar S and Praveenkumar C},
  title = {{KEC\_TECH\_TITANS@DravidianLangTech 2025: Abusive Text Detection in Tamil and Malayalam Social Media Comments Using Machine Learning}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure