TL;DR
This paper tackles the challenging problem of detecting hate speech in Dravidian languages (Tamil, Telugu, Malayalam) by fusing text and audio modalities. They found that a Random Forest model excels at text-based detection, while a Convolutional Neural Network (CNN) is best for audio, and their combined approach shows promise for more robust content moderation.
What Problem Does This Paper Address?
The proliferation of social media has brought hate speech to the forefront of online content moderation challenges. While text-based detection has seen significant advancements, spoken content (audio messages, voice notes) also plays a crucial role in spreading harmful discourse. This is particularly complex in multilingual communities, where code-mixing, informal language, and phonetic variations in languages like Tamil, Telugu, and Malayalam create unique hurdles for automated detection systems. Existing methods often focus on a single modality, leaving a gap in robust, multimodal hate speech detection for these under-resourced Dravidian languages.
Methods
The authors propose a multimodal approach that leverages both text and audio features. For text preprocessing, they employ standard techniques like character removal, tokenization, normalization, and TF-IDF representation, with SMOTE used to address class imbalance. For audio preprocessing, they focus on volume normalization, noise reduction, resampling, silence trimming, and phoneme segmentation. Key acoustic features like MFCCs and Mel Spectrograms are extracted.
They then evaluate several machine learning and deep learning models for classification:
- Text Classification: Random Forest, Bi-directional LSTM (BiLSTM), Multinomial Naive Bayes, and Capsule Networks with Attention-based BiLSTM.
- Audio Classification: Convolutional Neural Network (CNN).
The core of their approach involves feature fusion, where the outputs of the best-performing text model (Random Forest) and audio model (CNN) are combined to improve overall hate speech detection accuracy.
Datasets
The study utilizes a multimodal dataset specifically curated for hate speech detection in Tamil, Telugu, and Malayalam, sourced from social media platforms. This dataset contains both text and audio content, with instances categorized into "Hate" and "Non-Hate" classes. The "Hate" class is further subdivided into four types: Gender (G), Political (P), Religious (R), and Personal Defamation (C). The paper provides data distribution tables for both text and audio.
Key Insights and Ideas
- Modality Matters: The research strongly suggests that a multimodal approach is superior to single-modality detection for hate speech in Dravidian languages. Combining text and audio captures a more comprehensive understanding of potentially harmful content.
- Classical Models Shine for Text: Surprisingly, the relatively simpler Random Forest model outperformed more complex deep learning architectures like BiLSTM for text-based hate speech detection in this context. This highlights the importance of robust feature engineering and ensemble learning for handling noisy, real-world text data.
- CNNs are Audio Champs: Convolutional Neural Networks (CNNs) proved to be highly effective for audio-based hate speech detection, adept at capturing complex temporal and frequency patterns that other models struggled with.
- Complexity Can Hurt: More intricate models like ConvLSTM + BiLSTM, while theoretically capable of capturing complex dependencies, suffered from overfitting on the training data, leading to poorer performance compared to simpler models.
Key Results
- Text Classification: Random Forest achieved the highest accuracy across all languages: Tamil (0.9373), Telugu (0.8838), and Malayalam (0.8153).
- Audio Classification: CNN achieved good accuracy, with the highest in Malayalam (0.7577), followed by Telugu (0.7207) and Tamil (0.7059).
- Multimodal Performance: While the paper doesn't provide a single aggregated multimodal F1 score for their final fused model, their participation in the DravidianLangTech@NAACL 2025 shared task resulted in rankings of ninth in Tamil (F1: 0.3018), ninth in Telugu (F1: 0.251), and thirteenth in Malayalam (F1: 0.2782). These scores, while modest, indicate progress in a challenging domain.
Limitations and Future Work
The authors acknowledge several limitations:
- Fusion Strategy: The current fusion strategy could be improved, potentially by exploring transformer-based solutions.
- Performance Metrics: The F1 scores, particularly for Malayalam, are noted as needing improvement.
- Model Complexity: Complex models like ConvLSTM and BiLSTM underperformed, suggesting a need for more careful architecture design or regularization.
- Reproducibility: They suggest that future work should focus on guaranteeing reproducibility through open-source implementation.
Future research directions include enhancing fusion techniques, exploring more advanced models, and ensuring open-source availability for broader adoption and further research.
Summary updated