TL;DR
This paper tackles the critical issue of online abuse targeting women on social media, specifically within the Tamil and Malayalam languages. By employing a Long Short-Term Memory (LSTM) deep learning model, the researchers developed a system capable of detecting abusive content in code-mixed text, achieving promising F1 scores for both languages. This work highlights the potential of deep learning for abuse detection in resource-constrained languages.
What Problem Does This Paper Address?
The digital age has unfortunately amplified online violence, with women disproportionately affected by gendered abuse. While online abuse detection systems exist for languages like English, there's a significant gap in addressing this issue for Dravidian languages such as Tamil and Malayalam. These languages, spoken by a vast population on social media, present unique challenges due to their informal nature, including slang, abbreviations, code-mixing, and code-switching. The casual and diverse linguistic landscape makes it difficult for existing models to accurately identify and combat abusive content, leading to a lack of effective protective measures for women in these linguistic communities.
Methods
The core of this research lies in the application of a Long Short-Term Memory (LSTM) network for abusive text detection. LSTMs are a type of recurrent neural network (RNN) particularly adept at handling sequential data like text, allowing them to capture long-range dependencies and contextual relationships between words.
The methodology followed these steps:
- Dataset Gathering: A dataset of abusive and non-abusive user-generated content from YouTube was compiled, specifically focusing on code-mixed text (Malayalam/English and Tamil/English). This dataset was sourced from the DravidianLangTech@NAACL2025 shared task.
- Preprocessing: Standard NLP preprocessing techniques were applied, including converting text to lowercase, removing unwanted spaces, and tokenization. Tokenized text was then padded to ensure uniform input shape for the model.
- Model Architecture: The LSTM model comprised an embedding layer (mapping words to dense vectors), the LSTM layer itself (to maintain sequential relationships), and a dense output layer with a sigmoid activation function for binary classification (abusive vs. non-abusive).
- Training and Optimization: The model was trained using the Adam optimizer with binary cross-entropy loss. It was trained for five epochs with a batch size of 64. Dropout was employed to mitigate overfitting.
- Evaluation: The model's performance was evaluated using the F1-score, a metric that balances precision and recall, on unseen test data after each epoch.
Datasets
The study utilized two code-mixed datasets:
- Malayalam code-mixed: Text containing both Malayalam and English words.
- Tamil code-mixed: Text containing both Tamil and English words.
These datasets were categorized as abusive or non-abusive and were part of the DravidianLangTech@NAACL2025 shared task. Table 1 provides a statistical breakdown of the training and validation data for both languages.
Key Insights and Ideas
- Deep Learning's Efficacy in Resource-Constrained Languages: The research demonstrates that deep learning models like LSTMs can be effectively applied to detect abusive content even in languages with limited annotated data, such as Tamil and Malayalam.
- Code-Mixing as a Significant Challenge: The prevalence of code-mixing (interspersing words from different languages) in social media text poses a substantial hurdle for traditional text classification models, making specialized approaches necessary.
- Targeting Gendered Abuse: The paper specifically focuses on abuse directed at women, highlighting the need for nuanced detection systems that can identify the subtle and often harmful ways this abuse manifests online.
- Importance of Language-Specific Nuances: The authors acknowledge that while Tamil and Malayalam are both Dravidian languages, their unique morphological, syntactic, and semantic structures require language-specific handling for optimal detection.
Key Results
The LSTM model achieved promising results on the code-mixed datasets:
- Malayalam: An F1 score of 0.7571, ranking the model 1st in the shared task for this language.
- Tamil: An F1 score of 0.7207, ranking the model 18th in the shared task for this language.
These scores indicate a good balance between identifying abusive content and avoiding false positives, showcasing the model's effectiveness.
Limitations and Future Work
The authors acknowledge several limitations and suggest future directions:
- Dataset Size: The primary limitation is the inadequacy of the annotated dataset for both Malayalam and Tamil. Larger, more comprehensive datasets would significantly improve model effectiveness and generalizability.
- Informal Language Complexity: The informal nature of social media language, including slang, regional variations, and shortened expressions, presents ongoing challenges. The lack of clear grammatical structure hinders sentiment determination.
- Model Improvements: Future work could explore more complex neural network architectures, incorporate additional features like sentiment analysis, and leverage larger and more varied datasets.
- False Positives: The model faces challenges with false positives, requiring further attention and improvement in its discrimination capabilities.
- Optimization: Investigating techniques to reduce false positive identification and exploring the integration of large language models for optimization are suggested.
Summary updated