Misogynistic Meme Detection in Dravidian Languages Using Kolmogorov Arnold-based Networks

Manasha Arunachalam, Navneet Krishna Chukka, Harish Vijay V, Premjith B, Bharathi Raja ChakravarthiNAACL 2025

TL;DR

This paper introduces a novel pipeline for detecting misogynistic memes in Tamil and Malayalam, combining advanced image and text embeddings with three types of Kolmogorov-Arnold Networks (KANs). The research highlights the effectiveness of these multimodal approaches, particularly Chebyshev KAN for Tamil and FastKAN for Malayalam, in addressing the challenges of detecting harmful content in low-resource languages.

What Problem Does This Paper Address?

The proliferation of misogynistic content online poses a significant threat to creating safe and inclusive digital spaces, especially for women. While research has begun to address this issue, a critical gap exists in effectively detecting such harmful content within memes, particularly in low-resource languages like Tamil and Malayalam. These languages present unique challenges due to cultural nuances that influence the interpretation of both text and images. Existing methods often focus on single modalities or struggle to integrate them effectively, leading to suboptimal performance. This paper aims to bridge this gap by developing a robust multimodal system capable of accurately identifying misogynistic memes in these Dravidian languages.

Methods

The core of this research lies in a multimodal approach that fuses image and text representations before feeding them into advanced classification models.

  • Image Embeddings: The Vision Transformer Encoder (ViTE) model is employed to extract rich, contextual image embeddings. ViTE breaks down images into patches, linearly transforms them, and then uses positional embeddings and transformer encoder layers to capture both local and global visual features.
  • Text Embeddings: ModernBERT, an enhanced BERT architecture with an extended context length (8,192 tokens) and Rotary Positional Embeddings (ROPE), is used to generate semantically meaningful text embeddings from meme transcriptions.
  • Multimodal Fusion: The image and text embeddings are fused to create a single, comprehensive representation for each meme.
  • Kolmogorov-Arnold Networks (KANs): Three variants of KANs are utilized for classification:
  • PyKAN: A standard KAN implementation.
  • Chebyshev KAN: Leverages Chebyshev polynomials for function approximation, aiming for improved accuracy and efficiency over standard KAN.
  • FastKAN: Substitutes B-spline bases with Gaussian Radial Basis Functions (RBFs) to significantly reduce computational cost, making it suitable for real-time applications.

Datasets

The study utilizes the Misogyny Meme Detection dataset, which comprises images, corresponding transcriptions, and binary labels (0 for non-misogynistic, 1 for misogynistic). This dataset is available for both Tamil and Malayalam.

  • Tamil Dataset: Includes training, development, and test sets with a specific number of images and corresponding CSV files for transcriptions and labels. The paper notes efforts to ensure dataset uniqueness and integrity by cross-checking and reassigning missing images.
  • Malayalam Dataset: Also includes training, development, and test sets with a similar structure.

Key Insights and Ideas

  • Multimodal Fusion is Crucial: The research strongly supports the idea that combining visual and textual information is essential for accurately detecting misogynistic memes, especially in culturally nuanced contexts.
  • KANs Show Promise for Low-Resource Languages: The application of Kolmogorov-Arnold Networks, particularly Chebyshev KAN and FastKAN, demonstrates their potential for effective classification in languages with limited annotated data.
  • Class Imbalance is a Major Hurdle: The underrepresentation of misogynistic instances (class 1) significantly impacts model performance, highlighting the need for specialized techniques to address this.
  • Language-Specific Performance Differences: While both Tamil and Malayalam datasets were used, the optimal KAN model varied, with Chebyshev KAN excelling in Tamil and FastKAN in Malayalam, suggesting that language-specific characteristics influence model choice.

Key Results

  • Tamil Results: Chebyshev KAN achieved the highest F1-score of 0.77, followed by PyKAN (0.76) and FastKAN (0.73). All models showed good performance on class 0 (non-misogynistic) but struggled more with class 1 (misogynistic).
  • Malayalam Results: FastKAN achieved the highest accuracy (0.88) and F1-score (0.87). Chebyshev KAN and PyKAN followed with accuracies of 0.87 and 0.86, respectively. ROC curves for all models indicated excellent discrimination capabilities with AUC values exceeding 0.85.
  • Class 1 Performance: Across both languages, models generally exhibited higher precision and recall for class 0. Recall for class 1 was consistently lower, indicating a challenge in identifying positive (misogynistic) instances.
  • Oversampling Benefits: Techniques like SMOTE were employed to mitigate class imbalance, leading to improved performance metrics for the misogynistic class.

Limitations and Future Work

  • Limitations:
    • Class Imbalance: Despite oversampling techniques, the inherent class imbalance in the dataset remains a challenge, particularly affecting the recall for misogynistic content.
  • Model Performance on Class 1: All models struggled to achieve high recall for class 1, indicating room for improvement in identifying misogynistic instances.
  • Future Work:
    • Expand to More Languages: Extend the approach to other Indian languages to broaden its applicability.
  • Improve Fusion Techniques: Explore more advanced multimodal fusion methods for enhanced accuracy.
  • Real-time Deployment: Optimize models for real-time performance in social media moderation tools.
  • Bias Detection and Explainability: Incorporate techniques to address potential biases and improve model transparency and fairness.
  • Further Tuning: Fine-tuning models and data adjustments are suggested to enhance overall performance, especially for class 1 detection.

Summary updated

BibTeX
@inproceedings{arunachalam2025misogynistic,
  author = {Manasha Arunachalam and Navneet Krishna Chukka and Harish Vijay V and Premjith B and Bharathi Raja Chakravarthi},
  title = {{Misogynistic Meme Detection in Dravidian Languages Using Kolmogorov Arnold-based Networks}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure