TL;DR
This paper tackles the critical issue of identifying abusive content targeting women on social media, specifically focusing on low-resource Dravidian languages like Tamil and Malayalam. By adapting a powerful pre-trained multilingual model (mBERT), the researchers achieved promising results in classifying this harmful content, highlighting the need for specialized tools in under-represented linguistic contexts.
What Problem Does This Paper Address?
The pervasive nature of social media has unfortunately led to an increase in online harassment and hate speech. While significant research has focused on English, many low-resource languages, particularly Dravidian languages like Tamil and Malayalam, remain under-served in terms of tools to detect and combat such harmful content. This paper specifically addresses the challenge of identifying abusive content directed at women on social media in these languages, a crucial area given the disproportionate impact of online abuse on women. The authors aim to bridge this gap by developing a classification system that can accurately distinguish between abusive and non-abusive content in Tamil and Malayalam, including code-mixed text.
Methods
The core of this research lies in leveraging a pre-trained transformer model, specifically BERT base multilingual cased (mBERT). This model is chosen for its ability to handle multiple languages simultaneously, making it suitable for the multilingual nature of the dataset. The authors employ a standard supervised learning approach.
The key steps involved are:
- Preprocessing: This crucial stage involves cleaning the raw text data. Techniques used include:
- Removal of noise (special symbols, punctuations, emojis).
- Tokenization (breaking sentences into words).
- Lemmatization (reducing words to their root form).
- Removal of stop words (common, insignificant words).
- Case conversion (converting text to lowercase).
- Model Training: The mBERT model is fine-tuned on the preprocessed dataset. The training parameters include:
- Epochs: 7
- Max length: 256
- Batch size: 16
- Optimizer: AdamW
- Learning rate: 2e-5
- Correct bias: True (with a note that setting it to False for Malayalam yielded biased results).
- Classification: The fine-tuned mBERT model is then used to classify new instances as either "Abusive" (P=1) or "Non-Abusive" (P=0).
Datasets
The dataset used in this study was provided by DravidianLangTech for the shared task "Abusive Tamil and Malayalam Text targeting Women on Social Media" at DravidianLangTech@NAACL 2025. This dataset consists of YouTube comments in Tamil and Malayalam, annotated as either abusive or non-abusive. The paper provides descriptions of the dataset distribution for both Tamil and Malayalam, including training, validation, and testing splits.
Key Insights and Ideas
- The Power of Multilingual Models for Low-Resource Languages: The study demonstrates that pre-trained multilingual models like mBERT can be effectively adapted for low-resource languages, even with limited data, to tackle complex NLP tasks like hate speech detection.
- Importance of Language-Specific Preprocessing: The authors note that certain preprocessing steps, like lemmatization, can have different impacts on different languages. For instance, lemmatization negatively affected the accuracy for Tamil in their experiments.
- The Nuance of "Abusive" Content: The definition of "abusive" content is specifically tailored to target women, encompassing violence, abuse, mistreatment, or neglect, highlighting the need for context-aware classification.
- Challenges in Code-Mixed Text: While not explicitly detailed, the mention of "code mix" in the abstract suggests that the model needs to handle instances where Tamil and Malayalam are mixed within the same text, a common phenomenon in real-world social media.
Key Results
The mBERT model achieved the following performance metrics on the development dataset:
- Tamil:
- Accuracy: 0.77
- Precision: 0.78
- F1 score: 0.77
- Malayalam:
- Accuracy: 0.72
- Precision: 0.72
- F1 score: 0.72
For the shared task itself, the model achieved an mF1 score of 0.765 for Tamil and 0.677 for Malayalam. Confusion matrices are provided for both languages, visually illustrating the model's performance in correctly and incorrectly classifying instances.
Limitations and Future Work
The authors acknowledge several limitations:
- Small Training Dataset Size: The limited size of the training dataset restricts the model's generalization capabilities, potentially leading to struggles with unseen data.
- Ambiguity and Morphological Complexity: The inherent ambiguity of words and the morphological complexity common in low-resource languages like Tamil and Malayalam pose challenges that need further addressing.
Future research directions suggested include:
- Preprocessing with Back Translation: Exploring back translation of sentences as a preprocessing technique to potentially improve accuracy.
- Implementation of Vectorization: Investigating the implementation of vectorization methods to enhance model performance.
- Addressing Linguistic Nuances: Further research is needed to specifically address the ambiguity and morphological complexity of these languages.
Summary updated