TL;DR
This paper introduces "Incepto," a novel multilingual model that effectively detects abusive content in Tamil and Malayalam, two low-resource languages often used in online harassment targeting women. By leveraging transfer learning with XLM-RoBERTa and multi-head attention, the model achieves competitive results, highlighting the potential of such approaches for improving online safety in underrepresented linguistic communities.
What Problem Does This Paper Address?
Social media has become a double-edged sword: a powerful tool for connection and information sharing, but also a breeding ground for harmful behavior, particularly online abuse targeting women. This abuse, often fueled by societal biases, can have severe psychological and social consequences. While significant research exists for high-resource languages, detecting and mitigating this abuse in low-resource languages like Tamil and Malayalam presents a substantial challenge. These languages often suffer from a lack of readily available linguistic resources, tools, and annotated datasets, making it difficult to train robust machine learning models. Furthermore, code-mixing (the interspersion of multiple languages within a single utterance) adds another layer of complexity to text analysis in these communities.
Methods
The researchers developed a multilingual model named "Incepto" that tackles the challenges of abusive content detection in Tamil and Malayalam. Their approach is rooted in transfer learning, a technique where a model pre-trained on a large dataset (in this case, XLM-RoBERTa, a powerful multilingual transformer) is adapted for a specific task.
Key methodological components include:
- XLM-RoBERTa-base: This pre-trained model serves as the foundation, providing strong initial language understanding capabilities across multiple languages.
- Multi-Head Attention: To better capture task-specific nuances relevant to abusive language, the model incorporates four multi-head attention layers. These layers allow the model to dynamically focus on different parts of the input text that are most indicative of abuse.
- Layer Normalization: Applied to the dense layers, this technique helps stabilize and improve the learning process.
- Softmax Layer: This final layer outputs the classification probabilities, indicating whether a comment is abusive or not.
The model architecture is designed to be efficient and effective, leveraging existing powerful pre-trained models while adding specialized layers for improved performance on the target task.
Datasets
The study utilizes two datasets for training and evaluation, both sourced from Priyadharshini et al. (2023, 2022):
- Tamil Dataset: Consists of 2790 comments, with approximately 1425 non-abusive and 1375 abusive samples in the training set. The development and test sets each contain 598 samples.
- Malayalam Dataset: Comprises 2933 comments, with roughly 1525 non-abusive and 1400 abusive samples in the training set. The development and test sets each contain 629 samples.
Both datasets are annotated with binary labels: "Abusive" and "Non-Abusive." The paper also includes visualizations of comment length distributions and word clouds for both languages to provide insights into the data characteristics.
Key Insights and Ideas
- Multilingual Transfer Learning is Promising for Low-Resource Languages: The success of Incepto demonstrates that leveraging powerful pre-trained multilingual models like XLM-RoBERTa can significantly boost performance in languages with limited resources, even with code-mixing.
- Multi-Head Attention Enhances Abusive Language Detection: The inclusion of multi-head attention layers proved crucial for the model to effectively identify subtle cues and patterns indicative of abusive language within the text.
- Code-Mixing is a Significant Hurdle: The paper implicitly highlights the challenges posed by code-mixed text, which is common in social media. The model's performance, while good, suggests that further advancements are needed to fully address this complexity.
- Reproducibility is Key: By releasing their model as a Python package and making the source code publicly available, the authors contribute to the research community, enabling others to build upon their work and fostering collaboration.
Key Results
The Incepto model achieved the following performance metrics:
- Tamil Dataset: A macro F1 score of 0.7864, securing 3rd place out of 27 teams in the DravidianLangTech 2025 leaderboard. This performance was very close to the top-ranked team.
- Malayalam Dataset: A macro F1 score of 0.7058, ranking 4th out of 35 teams. While competitive, this result indicates more room for improvement compared to Tamil.
These results underscore the model's effectiveness, particularly in Tamil, and its competitive capabilities in Malayalam, despite the inherent challenges of the language.
Limitations and Future Work
The authors acknowledge several limitations:
- Dataset Inadequacy: The annotated datasets for both Tamil and Malayalam are considered insufficient, impacting model effectiveness and generalizability.
- Informal Language: The informal nature of social media language, including regional variations, shortened expressions, and grammatical inconsistencies, poses a significant challenge for preprocessing and classification.
- Linguistic Differences: Despite belonging to the same language family, Tamil and Malayalam have significant differences in morphology, syntax, and semantics, complicating text analysis.
- Tokenizer Absence: The lack of a proper tokenizer specifically for Tamil and Malayalam makes text processing and model training more complex.
Future work suggested by the authors includes focusing on improving performance, especially for Malayalam, and potentially exploring more advanced techniques to address the identified limitations. The release of their model and code aims to facilitate further research and development in this critical area.
Summary updated