YenCS@DravidianLangTech 2025: Integrating Hybrid Architectures for Fake News Detection in Low-Resource Dravidian Languages

Anusha M D Gowda, Parameshwar R HegdeNAACL 2025

TL;DR

This paper tackles the challenging problem of fake news detection in low-resource Dravidian languages, which are often overlooked in NLP research. The authors propose hybrid deep learning models, combining recurrent neural networks (RNNs) and Long Short-Term Memory (LSTM) networks, to improve detection accuracy for both binary (fake vs. real) and fine-grained classification tasks. Their findings highlight the effectiveness of these hybrid approaches, especially for the more straightforward binary classification, while also pointing to the difficulties in nuanced multi-class detection.

What Problem Does This Paper Address?

Detecting fake news is a critical task in today's information-saturated world, but it's particularly challenging for languages with limited annotated data and complex linguistic structures, such as code-mixed Dravidian languages. Manually verifying the authenticity of news is a monumental task, and the rapid spread of misinformation poses significant risks to society. Existing research often focuses on high-resource languages, leaving a gap in effective fake news detection for under-resourced language communities. This paper aims to bridge that gap by exploring advanced hybrid deep learning architectures tailored for these specific linguistic challenges.

Methods

The researchers employed a two-pronged approach using hybrid deep learning models.

  • Pre-processing: Standard text cleaning techniques were applied, including stopword removal, punctuation and digit elimination, and tokenization using Keras Tokenizer. Sequences were then padded to a uniform length of 100 words, and labels were one-hot encoded for multi-class tasks.
  • Feature Extraction: FastText embeddings were utilized to represent words as dense vectors, capturing semantic meaning and enabling better generalization, even for out-of-vocabulary words, by leveraging character n-grams.
  • Model Architectures:
    • Binary Classification (Task A): A "Fusion Model" was developed, integrating predictions from a Bidirectional RNN (Bi-RNN) and an LSTM-RNN model through weighted averaging. This ensemble approach aims to leverage the strengths of both architectures.
  • Fine-grained Classification (Task B): Several models were explored, including GRU, LSTM, and a hybrid LSTM+GRU model. These models were designed to capture sequential dependencies in the text for multi-class categorization.

Datasets

The study utilized datasets from the Shared Task on Fake News Detection in Dravidian Languages, organized at Dravidian-LangTech@NAACL 2025. The paper provides class-wise distributions for two tasks:

  • Task A (Binary Classification): A dataset with "Original" and "Fake" news categories.
  • Task B (Fine-grained Classification): A dataset with five categories: "Half True," "False," "Partly False," and "Mostly False."

The paper does not explicitly state if these datasets are newly introduced benchmarks, but it emphasizes their importance for advancing fake news detection in the under-explored Dravidian language domain.

Key Insights and Ideas

  • Hybrid Models Shine for Binary Tasks: The fusion of Bi-RNN and LSTM-RNN models proved highly effective for the binary classification task, suggesting that ensemble methods can significantly boost performance when distinguishing between clear-cut categories like "fake" and "original."
  • Multi-class is a Tough Nut to Crack: Detecting nuanced categories like "Half True" or "Partly False" is considerably more challenging. The models struggled to achieve high accuracy in Task B, indicating that current architectures may not fully capture the subtle semantic differences required for such fine-grained distinctions.
  • Low-Resource Languages Need Tailored Solutions: The study underscores the need for specialized approaches for languages with limited data. The effectiveness of FastText embeddings, which can handle out-of-vocabulary words better than standard Word2Vec, is a testament to this.
  • Class Imbalance is a Major Hurdle: The paper explicitly mentions class imbalance as a significant factor hindering performance in multi-class classification, a common issue in real-world datasets.

Key Results

  • Task A (Binary Classification): The Fusion Model achieved a strong macro-average F1-score of 0.792 on the development set. Other models like Bi-RNN and LSTM-RNN Hybrid also showed promising results, with F1-scores around 0.79 and 0.82 respectively.
  • Task B (Fine-grained Classification): Performance was significantly lower. The LSTM+GRU Hybrid model achieved a macro-average F1-score of 0.26 on the development set. Individual models like GRU and LSTM also reported lower F1-scores (e.g., 0.63 and 0.60 respectively).

Limitations and Future Work

  • Limitations:
    • Multi-class Difficulty: The primary limitation highlighted is the poor performance in fine-grained (multi-class) classification, attributed to class imbalance and the models' inability to discern subtle semantic differences.
  • Contextual Understanding: Sequential models, while good at capturing dependencies, may lack a deep understanding of context, making it harder to detect sarcasm or implicit misinformation.
  • Computational Cost & Deployment: For low-resource environments, the high computational cost of deep learning models can limit real-time deployment.
  • Interpretability: The "black-box" nature of deep learning models impacts interpretability and trust.
  • Future Work:
    • Class Balancing: Techniques to address class imbalance are crucial for improving multi-class performance.
  • Advanced Embeddings: Exploring more powerful embeddings like BERT or XLM-R could enhance contextual understanding.
  • Attention Mechanisms & Transformers: Integrating attention mechanisms and transformer-based architectures is suggested to improve contextual encoding.
  • Language-Specific Preprocessing: Developing preprocessing methods tailored to Dravidian languages could further boost performance.
  • Explainability: Investigating explainability strategies and attention-based models is recommended to build trust.

Summary updated

BibTeX
@inproceedings{gowda2025yencs,
  author = {Anusha M D Gowda and Parameshwar R Hegde},
  title = {{YenCS@DravidianLangTech 2025: Integrating Hybrid Architectures for Fake News Detection in Low-Resource Dravidian Languages}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure