TL;DR
This paper tackles the challenging problem of detecting hate speech across multiple Dravidian languages (Tamil, Telugu, Malayalam) using both text and audio from social media. Their multimodal approach, leveraging advanced preprocessing and deep learning models like XLM-Roberta and CNNs, achieved competitive results in a recent shared task, highlighting the importance of these techniques for under-resourced languages.
What Problem Does This Paper Address?
Social media has democratized information sharing, but it has also amplified hate speech, posing significant societal challenges. Detecting hate speech is crucial, especially in low-resource languages like Tamil, Telugu, and Malayalam, which present unique difficulties due to their linguistic complexity and the prevalence of code-mixing with English. Traditional NLP methods often struggle with these languages, and social media content is inherently multimodal (text, audio, images). This paper addresses the need for effective multimodal hate speech detection systems specifically for these Dravidian languages, a task that is complicated by limited data and computational resources.
Methods
The authors employed a comprehensive multimodal approach. For text preprocessing, they utilized IndicNormalizerFactory for normalization, 'indic_tokenize.trivial_tokenize' for tokenization, and custom stopword lists. Data augmentation was performed using nlpaug for synonym replacement and word insertion. Feature extraction for text involved TF-IDF, Count Vectorizer, and transformer embeddings, with XLM-Roberta-Large showing the best performance.
For audio preprocessing, Mel Frequency Cepstral Coefficients (MFCCs) were extracted. Augmentation techniques like noise addition, time-stretching, and pitch-shifting were applied to improve model robustness. A Convolutional Neural Network (CNN) was used for audio classification.
The overall model architecture consists of separate text and audio classification models. The text model uses XLM-Roberta-Large or BERT-Multilingual Cased embeddings, followed by dense layers, batch normalization, and dropout, culminating in a softmax output for classification into categories like Gender, Political, Religious, Personal Defamation, and Non-Hate. The audio model processes MFCC features through convolutional layers and a dense layer before the softmax output. Optimization techniques like Adam, categorical cross-entropy, dropout, and early stopping were employed.
Datasets
The study utilized datasets provided for the DravidianLangTech@NAACL 2025 Multimodal Hate Speech Detection Shared Task. These datasets included text and audio components for Tamil, Telugu, and Malayalam, with hate speech categorized into Gender, Political, Religious, Personal Defamation, and Non-Hate. The paper provides distributions for text and audio data, showing the number of non-hate and hate instances for each language.
Key Insights and Ideas
- Multimodality is Key: The paper strongly emphasizes that combining text and audio is crucial for robust hate speech detection, especially on social media where content is rarely purely textual.
- XLM-Roberta's Strength: XLM-Roberta-Large emerged as a powerful text encoder, demonstrating strong multilingual capabilities that are vital for handling code-mixed Dravidian languages.
- Augmentation's Impact: Data augmentation techniques for both text and audio significantly improved model performance and robustness, helping to mitigate some of the challenges posed by limited datasets.
- Preprocessing Matters: Thorough preprocessing, including normalization and specialized tokenization for Indic languages, is foundational for effective feature extraction and model performance.
- Low-Resource Focus: The work highlights the critical need for developing and applying advanced NLP techniques to under-resourced languages, which often face unique challenges that require tailored solutions.
Key Results
The proposed multimodal system achieved competitive results in the DravidianLangTech@NAACL 2025 shared task:
- Text Classification: XLM-Roberta-Large achieved the highest training accuracies: 83% (Tamil), 88% (Telugu), and 85% (Malayalam).
- Audio Classification: A CNN-based model achieved training accuracies of 88% (Tamil), 88% (Telugu), and 93% (Malayalam).
- Overall Ranking: The team ranked third in Tamil (Macro F1: 0.6438), twelfth in Malayalam (Macro F1: 0.3016), and fifteenth in Telugu (Macro F1: 0.1559).
Limitations and Future Work
The authors acknowledge several limitations:
- Limited Dataset Size: The primary challenge is the small dataset size, which hinders the model's ability to generalize across diverse hate speech patterns.
- Class Imbalance: The dataset suffers from class imbalance, potentially leading to biased classification and under-representation of certain hate speech categories.
- Noisy Annotations: The presence of noisy annotations in the dataset also poses a challenge.
Future work suggested includes:
- Dataset Expansion: Creating larger and more diverse datasets.
- Advanced Balancing Techniques: Implementing more effective methods to address class imbalance.
- Exploring Additional Modalities: Incorporating other modalities like images.
- Specialized Architectures: Developing architectures specifically designed for multimodal tasks.
- Domain Adaptation: Investigating better domain adaptation strategies.
Summary updated