TL;DR
This paper presents a robust approach for detecting abusive comments in Tamil and code-mixed Tamil-English text, achieving top rankings in a shared task. By combining TF-IDF with character n-grams, Random Kitchen Sink for feature expansion, and a Support Vector Machine classifier, the team effectively tackles the challenges of noisy social media data and class imbalance.
What Problem Does This Paper Address?
The proliferation of online social media has unfortunately led to an increase in abusive and hateful content. Identifying and mitigating such content is crucial for fostering healthier online environments. This paper specifically addresses the challenge of detecting abusive comments in Tamil, a language with unique linguistic characteristics and a growing presence in digital discourse. Furthermore, it tackles the complexity of code-mixed Tamil-English comments, which are common in online interactions but pose significant challenges for traditional NLP models. The authors aim to build a system that can accurately classify these comments, contributing to the development of tools for content moderation and online safety in the Tamil language community.
Methods
The core of the CEN-Tamil approach lies in a multi-stage process designed to handle the nuances of social media text.
- Text Representation: They begin by using TF-IDF (Term Frequency-Inverse Document Frequency) with character word-bound n-grams. This is a standard technique for capturing word importance, but the use of character n-grams helps in handling variations in spelling and morphology common in informal text.
- Feature Expansion: To address potential limitations of the TF-IDF feature space and to better capture complex relationships, they employ Random Kitchen Sink (RKS). RKS is a method for mapping features into a higher-dimensional space, effectively approximating kernel methods without explicit kernel computation. This is particularly useful for dealing with sparse data.
- Handling Class Imbalance: Social media datasets often suffer from severe class imbalance, where abusive comments are far less frequent than non-abusive ones. To combat this, the authors utilize SMOTE (Synthetic Minority Over-sampling Technique). SMOTE generates synthetic samples for the minority class, helping to balance the dataset and prevent the classifier from being biased towards the majority class.
- Classification: Finally, a Support Vector Machine (SVM) classifier with a polynomial kernel is used for the actual classification. SVMs are known for their effectiveness in high-dimensional spaces, making them a suitable choice after the feature expansion by RKS.
Datasets
The research utilizes two datasets provided for the shared task:
- Tamil Dataset: This dataset contains comments exclusively in Tamil.
- Tamil-English Dataset: This dataset comprises code-mixed comments, where Tamil and English are interspersed.
These datasets are divided into training, validation, and test sets. The paper notes that these datasets are highly imbalanced, which informed the decision to use SMOTE.
Key Insights and Ideas
- The Power of Character N-grams: The use of character word-bound n-grams in TF-IDF proved effective in capturing the noisy and varied nature of social media text in Tamil, outperforming simpler word-based approaches.
- RKS for Feature Enhancement: Random Kitchen Sink played a crucial role in expanding the feature space, allowing the SVM classifier to better discern complex patterns in the text, especially when dealing with high-dimensional representations.
- SMOTE's Critical Role: The significant class imbalance in the datasets made SMOTE an indispensable component for achieving competitive performance. Without it, the model would likely have been biased towards the majority class.
- SVM's Robustness: The SVM classifier, with its polynomial kernel, demonstrated strong performance in classifying the expanded feature sets, highlighting its suitability for this type of text classification task.
Key Results
The CEN-Tamil team achieved significant success in the shared task:
- Rank 1 for Tamil: Their approach secured first place in the Tamil abusive comment detection task, achieving an F1-score of 0.32.
- Rank 7 for Tamil-English: For the code-mixed Tamil-English dataset, they secured seventh place with an F1-score of 0.25.
The paper highlights that their final model (Model-4), which combined TF-IDF, RKS, and SMOTE with SVM, consistently outperformed simpler models that omitted one or more of these components. For the Tamil test dataset, the "None-of-the-above" class had the highest F1-score (0.83), while the "Transphobic" class had the lowest (0.00) due to very few data points. For the Tamil-English test dataset, "None-of-the-above" also had the highest F1-score (0.85), and "Misogyny" had the lowest (0.18).
Limitations and Future Work
The authors acknowledge several limitations and propose future directions:
- Limited Data for Some Classes: The "Transphobic" class in the Tamil dataset had very few data points, leading to a zero F1-score, indicating a challenge in learning from such sparse data.
- Exploration of Transformers: The current approach does not explore advanced deep learning architectures like Transformers. The authors suggest that future work should involve experimenting with models such as BERT, LaBSE, LSTM, and CNN to potentially further improve performance. This would allow for a deeper understanding of contextual nuances in the text.
Summary updated