HTMS@DravidianLangTech 2025: Fusing TF-IDF and BERT with Dimensionality Reduction for Abusive Language Detection in Tamil and Malayalam

Bachu Naga Sri Harini, Kankipati Venkata Meghana, Kondakindi Supriya, Tara Samiksha, Premjith BNAACL 2025

TL;DR

This paper tackles the challenging task of detecting abusive language in Tamil and Malayalam social media comments, particularly those targeting women. The authors propose a hybrid embedding approach that fuses TF-IDF and BERT, along with dimensionality reduction, to improve detection accuracy. Their findings highlight the effectiveness of this fused approach, outperforming individual TF-IDF or BERT models.

What Problem Does This Paper Address?

Social media platforms are rife with toxic content, and detecting abusive language is a significant challenge. This is especially true for low-resource languages like Tamil and Malayalam, where data imbalance, code-mixing, and the nuanced nature of abusive language make automated detection difficult. The paper specifically focuses on abusive comments targeting women on social media, a critical area for online safety. Existing methods often struggle to capture the full spectrum of abusive language, leading to inaccurate classifications.

Methods

The researchers experimented with three distinct methods for abusive language detection:

  • TF-IDF Embedding-based Method: This approach uses Term Frequency-Inverse Document Frequency (TF-IDF) to represent words based on their importance in a document and across a corpus. They limited the feature set to 5,000 words and trained a logistic regression classifier. This served as one of their baseline submissions.
  • BERT-based Method: This method leverages the power of pre-trained BERT models ('bert-base-uncased') to generate contextual sentence embeddings. They processed comments through BERT, extracted representations from the last hidden layer, and averaged them to get sentence embeddings. These embeddings were then used to train a Random Forest classifier. This was another baseline submission.
  • Fused TF-IDF and BERT Embeddings: This is the core contribution. They combined TF-IDF and BERT embeddings. To address the significant dimensionality differences between TF-IDF (potentially very high) and BERT (fixed at 768 dimensions), they employed the Random Kitchen Sink (RKS) algorithm to reduce the dimensionality of TF-IDF embeddings to match BERT's dimension. These fused embeddings were then used to train a Random Forest classifier. This hybrid approach was their primary submission.

Datasets

The study utilized a dataset provided for the DravidianLangTech@NAACL 2025 shared task on abusive Tamil and Malayalam texts targeting women on social media. This dataset included training, development (validation), and test splits. The authors combined the training and development sets for model training. Table 1 provides a breakdown of the data points per split for Tamil and Malayalam.

Key Insights and Ideas

  • The Power of Fusion: Combining TF-IDF and BERT embeddings proved more effective than using either method alone. This suggests that capturing both word frequency/distribution (TF-IDF) and contextual meaning (BERT) is crucial for robust abusive language detection.
  • Dimensionality Reduction is Key: The success of the fused approach hinges on effectively managing the dimensionality mismatch between TF-IDF and BERT. The RKS algorithm provided a viable solution for this.
  • BERT's Contextual Understanding: BERT embeddings significantly improved the model's ability to understand subtle language nuances and contextual relationships, leading to better performance.
  • TF-IDF's Limitations: While TF-IDF is computationally efficient, it struggles to capture deeper semantic meaning and context, leading to a tendency to misclassify abusive comments as non-abusive, as seen in their results.

Key Results

  • Best Performance: The fused TF-IDF and BERT embedding model achieved the most promising results, outperforming individual TF-IDF and BERT models.
  • BERT-only Performance: The BERT-only model also showed excellent results, demonstrating its strong capability in understanding abusive language.
  • TF-IDF Baseline: The TF-IDF-only model performed poorly, predominantly classifying comments as non-abusive, leading to highly inaccurate results.
  • Quantitative Scores: Table 2 shows macro-averaged Precision, Recall, and F1-scores for the different models. The TF-IDF + BERT model achieved an F1-score of 0.62, matching the TF-IDF-only model's F1-score but with better precision and recall. The BERT-only model achieved an F1-score of 0.59. Table 3 shows macro F1-scores for BERT embedding on the predicted dataset, with Tamil at 0.50 and Malayalam at 0.49.

Limitations and Future Work

  • Low-Resource Languages: The authors acknowledge that Tamil and Malayalam are low-resource languages, meaning they had limited data points for training. This constraint prevented the model from uncovering all necessary patterns for classification.
  • Feature Representation: The approach for generating feature representations was identified as a reason for some of the lower performance scores.
  • Future Directions: The authors suggest that fine-tuning pre-trained models or using more data could improve performance. They also propose exploring other dimensionality reduction techniques or different fusion strategies.

Summary updated

BibTeX
@inproceedings{harini2025htms,
  author = {Bachu Naga Sri Harini and Kankipati Venkata Meghana and Kondakindi Supriya and Tara Samiksha and Premjith B},
  title = {{HTMS@DravidianLangTech 2025: Fusing TF-IDF and BERT with Dimensionality Reduction for Abusive Language Detection in Tamil and Malayalam}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure