SSNCSE@DravidianLangTech 2025: Multimodal Hate Speech Detection in Dravidian Languages

Sreeja K, Bharathi BNAACL 2025

TL;DR

This paper tackles the challenging problem of detecting multimodal hate speech in low-resource Dravidian languages by proposing a novel approach that combines text and audio features. Their method leverages a weighted majority fusion technique to effectively integrate predictions from various classifiers, achieving promising results for Tamil, Malayalam, and Telugu.

What Problem Does This Paper Address?

Hate speech is a pervasive issue on social media, and its detection is crucial for fostering safer online environments. However, detecting hate speech in low-resource languages, particularly those spoken in Dravidian regions, presents significant challenges due to limited linguistic data and cultural nuances. Furthermore, hate speech is not confined to text; it can be conveyed through audio as well, making multimodal analysis essential. Existing research has largely focused on English, leaving a gap in effective hate speech detection for languages like Tamil, Malayalam, and Telugu, where the problem is amplified by the dynamic nature of social media and the potential for chatbots to learn and propagate harmful content.

Methods

The authors propose a systematic methodology for multimodal hate speech detection that involves several key stages:

  • Data Preprocessing: This crucial step involves cleaning and normalizing both textual and audio data to ensure consistency and remove noise. Textual data is tokenized and processed using language-specific methods, while audio data undergoes noise removal.
  • Class Balancing: To address the inherent imbalance in hate speech datasets, where non-hate instances often far outnumber hate instances, the researchers employ a class weight mechanism. This assigns higher importance to minority classes during training, preventing classifiers from being biased towards the majority class.
  • Feature Extraction: Diverse modalities are transformed into feature vectors suitable for machine learning models. For text, TF-IDF and Count Vectorizers are used. For audio, MFCC and Log-mel spectrograms are employed.
  • Classifier Modeling: A range of classifiers are trained for the multiclass classification task (gender, political, religious, personal defamation, and non-hate). These include Support Vector Machines (SVM), Random Forests (RF), Multi-layer Perceptrons (MLP), and Logistic Regression (LR).
  • Majority Fusion: A two-level majority fusion mechanism is used to combine predictions. First, predictions from multiple classifiers within each modality (text and audio) are fused using a majority voting strategy. Then, the fused predictions from text and audio modalities are combined again using another majority voting mechanism to produce the final prediction. This approach aims to leverage the strengths of different modalities and classifiers.

Datasets

The study utilizes data sourced from the Multimodal Social Media Data Analysis in Dravidian Languages (MSMDA-DL) dataset, provided by DravidianLangTech@NAACL 2025. This dataset comprises multimodal (text and audio) data for Tamil, Malayalam, and Telugu. Each language has a specific number of training samples (Tamil: 514, Malayalam: 863, Telugu: 556) and 50 test samples per language. The task is a multi-class classification problem with five categories: Gender (G), Political (P), Religious (R), Personal Defamation (C), and Non-Hate (N).

Key Insights and Ideas

  • Multimodal Fusion is Key: The paper strongly advocates for the necessity of combining text and audio modalities for more robust hate speech detection, as hate speech can be conveyed through both.
  • Class Weights Matter for Imbalance: The class weight mechanism proved effective in improving performance by giving more importance to minority classes, which is crucial for imbalanced datasets common in hate speech detection.
  • Majority Fusion for Robustness: The proposed two-level majority fusion technique effectively combines predictions from different classifiers and modalities, leading to better overall performance.
  • Dravidian Languages Present Unique Challenges: The research highlights the specific difficulties in detecting hate speech in low-resource Dravidian languages, emphasizing the need for culturally sensitive models and tailored approaches.

Key Results

The performance of the multimodal hate speech detection model was evaluated using a macro F1-score. The paper presents two sets of results:

  • Without Class Weights (Table 2): The F1-scores for Tamil, Malayalam, and Telugu were 0.47, 0.34, and 0.34, respectively.
  • With Class Weights (Table 3): The F1-scores significantly improved to 0.59 for Tamil, 0.52 for Malayalam, and 0.33 for Telugu.

These results demonstrate that the class weight mechanism significantly enhances the model's performance, particularly for Tamil and Malayalam.

Limitations and Future Work

The authors acknowledge several limitations:

  • Data Scarcity: The cruciality of obtaining sufficient and representative data for low-resource languages is recognized, and the model's generalizability and strength may be impacted by this constraint.
  • Lingering Biases: While class imbalance is addressed, biases in predictions may still exist to some extent, especially for minority classes.
  • Model Sophistication: The model becomes more complex as text and audio modalities are added, suggesting that further research into optimizing this integration is needed.

The paper suggests that the proposed fusion technique still requires additional testing before it can be used in real-world scenarios, indicating a direction for future work.

Summary updated

BibTeX
@inproceedings{k2025ssncse,
  author = {Sreeja K and Bharathi B},
  title = {{SSNCSE@DravidianLangTech 2025: Multimodal Hate Speech Detection in Dravidian Languages}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure