KEC_AI_GRYFFINDOR@DravidianLangTech 2025: Multimodal Hate Speech Detection in Dravidian languages

Kogilavani Shanmugavadivel, Malliga Subramanian, ShahidKhan S, Shri Sashmitha.S, Yashica SNAACL 2025

TL;DR

This paper tackles the challenging problem of detecting hate speech in code-mixed Dravidian languages (Tamil, Malayalam, Telugu) by proposing a multimodal approach that combines text and audio analysis. The authors demonstrate that integrating these modalities significantly improves detection accuracy, outperforming single-modality methods and achieving competitive rankings in a shared task.

What Problem Does This Paper Address?

The proliferation of social media has unfortunately led to an increase in hate speech. Detecting and mitigating this harmful content is crucial for fostering inclusive online spaces. However, this task becomes significantly more complex in multilingual and code-mixed environments, such as those found in Dravidian languages where users frequently blend local scripts with borrowed words. The scarcity of annotated datasets for these languages further exacerbates the challenge. This paper aims to address this gap by developing and evaluating multimodal models for hate speech detection in Tamil, Malayalam, and Telugu.

Methods

The researchers employed a combination of machine learning and deep learning techniques. For text processing, they utilized Count Vectorizer and TF-IDF for feature extraction, followed by classification models including Logistic Regression, Ridge Classifier, and Random Forest. For audio processing, they focused on extracting prosodic features like pitch and energy after noise reduction, normalization, and segmentation. A Convolutional Neural Network (CNN) was used for audio classification. The core of their approach lies in the multimodal integration, where text and audio modalities are processed separately and then their outputs are combined for classification. This modular design allows for future experimentation with additional features and models.

Datasets

The study utilized YouTube-sourced text and audio data for Tamil, Malayalam, and Telugu. The text data was categorized into "Hate" (subclasses: Gender, Political, Religious, Personal Defamation) and "Non-Hate." The training sets comprised 883 Malayalam, 1,397 Tamil, and 1,953 Telugu records. The audio dataset had a similar structure, with training sets containing 883 Malayalam, 509 Tamil, and 551 Telugu recordings. The paper notes that these datasets are limited for Dravidian languages.

Key Insights and Ideas

  • Multimodality is Key: The most significant insight is that combining text and audio information provides a richer context for hate speech detection, leading to improved accuracy compared to using either modality alone.
  • Language-Specific Model Strengths: Different models excel in different languages and modalities. For instance, Logistic Regression and Random Forest were strong for text in Tamil and Malayalam respectively, while CNNs showed promise for audio, particularly in Telugu.
  • Prosodic Features Matter: The success of Ridge Classifier for Tamil audio detection highlights the importance of spectral and prosodic features in capturing aggressive or hostile speech tones.
  • Code-Mixing Complexity: The paper implicitly acknowledges the inherent difficulty of code-mixed languages, which necessitates robust preprocessing and model architectures.

Key Results

The multimodal approach demonstrated strong performance across the Dravidian languages:

  • Tamil: Logistic Regression achieved the highest text Macro-F1 score of 0.97. Ridge Classifier was best for audio with a Macro-F1 score of 0.75.
  • Malayalam: Random Forest achieved the best text F1-score of 0.97. CNN was the best for audio with an F1-score of 0.69.
  • Telugu: Ridge Classifier achieved the best text F1-score of 0.89. CNN was the best for audio with an F1-score of 0.87.

In the shared task, the authors achieved the 12th rank for Tamil and the 7th rank for Malayalam and Telugu out of 145 teams.

Limitations and Future Work

The authors acknowledge several limitations:

  • Limited Labeled Datasets: The scarcity of annotated data for Dravidian languages is a significant hurdle.
  • Audio Model Performance: The complexity of prosodic features and insufficient audio samples impacted audio model performance.
  • Class Imbalance: The dataset's class imbalance may have affected the models' generalization capabilities.

Future work should focus on expanding datasets and refining models to address these limitations. Further improvements in feature extraction and model optimization are also suggested to enhance performance. The code for this shared task is available on GitHub.

Summary updated

BibTeX
@inproceedings{shanmugavadivel2025kec,
  author = {Kogilavani Shanmugavadivel and Malliga Subramanian and ShahidKhan S and Shri Sashmitha.S and Yashica S},
  title = {{KEC\_AI\_GRYFFINDOR@DravidianLangTech 2025: Multimodal Hate Speech Detection in Dravidian languages}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure