KCRL@DravidianLangTech 2025: Multi-Pooling Feature Fusion with XLM-RoBERTa for Malayalam Fake News Detection and Classification

Fariha Haq, Md. Tanvir Ahammed Shawon, Md Ayon Mia, Golam Sarwar Md. Mursalin, Muhammad Ibrahim KhanNAACL 2025

TL;DR

This paper introduces a novel Multi-Pooling Feature Fusion (MPFF) architecture, leveraging XLM-RoBERTa, to tackle the challenging task of fake news detection in Malayalam YouTube comments. Their approach significantly outperforms existing methods in both binary and multiclass classification, demonstrating the effectiveness of their fusion strategy for low-resource languages.

What Problem Does This Paper Address?

The rapid proliferation of misinformation on social media platforms poses a significant threat, especially in languages with limited computational resources. Manually verifying the authenticity of content is becoming increasingly difficult due to the sheer volume of data. This paper focuses on the specific challenge of detecting fake news within Malayalam YouTube comments, a task complicated by the language's linguistic complexity and the nuanced nature of social media discourse. They address two sub-tasks: distinguishing between "Original" and "Fake" news (binary classification) and categorizing content into four veracity levels: "FALSE," "HALF TRUE," "MOSTLY FALSE," and "PARTLY FALSE" (multiclass classification).

Methods

The core of this work is the proposed Multi-Pooling Feature Fusion (MPFF) architecture. This approach builds upon the powerful XLM-RoBERTa transformer model. The key innovation lies in how they extract and combine features:

  • XLM-RoBERTa Backbone: They utilize a pre-trained XLM-RoBERTa model, a multilingual transformer known for its strong performance across various languages.
  • Multi-Pooling Strategy: Instead of relying solely on the [CLS] token (a common practice for sentence representation), MPFF extracts features from three sources:
  • The [CLS] token itself.
  • The mean pooling of all token representations.
  • The max pooling of all token representations.
  • Feature Fusion: These three sets of features are then concatenated. This fused representation is fed into a classifier layer to make the final prediction. This fusion aims to capture a richer and more comprehensive understanding of the text's semantic and contextual nuances.

For preprocessing, they employ standard techniques like removing URLs, handling emojis, and normalizing punctuation. To address class imbalance in the multiclass task, they use random oversampling. They also compare their MPFF approach against several traditional Machine Learning (ML) models (LR, SVM, RF, XGBoost) using TF-IDF features and other Deep Learning (DL) architectures (LSTM, BiLSTM, BiGRU+CNN).

Datasets

The research utilizes datasets provided for the DravidianLangTech 2025 shared task on Fake News Detection in Malayalam YouTube comments.

  • Task 1 (Binary Classification): Consists of 5,091 texts, split into 3,257 for training, 815 for validation, and 1,019 for testing. The classes are "Original" and "Fake."
  • Task 2 (Multiclass Classification): Consists of 2,100 texts, split into 1,900 for training and 200 for testing. The classes are "FALSE," "HALF TRUE," "MOSTLY FALSE," and "PARTLY FALSE."

The paper notes that the multiclass dataset is relatively small and imbalanced, which presents a significant challenge.

Key Insights and Ideas

  • The Power of Feature Fusion: The MPFF architecture demonstrates that combining [CLS], mean, and max pooling from transformer embeddings is more effective than using any single source alone, especially for nuanced tasks like fake news detection.
  • XLM-RoBERTa's Strength in Low-Resource Settings: The success of XLM-RoBERTa, a multilingual model, highlights its potential for languages like Malayalam where large, language-specific pre-trained models might not be readily available.
  • Multiclass is Significantly Harder: The performance drop in multiclass classification compared to binary classification underscores the difficulty of distinguishing between subtle shades of misinformation.
  • Data Imbalance is a Major Hurdle: The authors explicitly mention the challenge posed by the imbalanced multiclass dataset, which impacts the model's ability to learn from minority classes.

Key Results

The MPFF approach with XLM-RoBERTa achieved state-of-the-art results on the DravidianLangTech 2025 dataset:

  • Task 1 (Binary Classification): Achieved a macro-averaged F1 score of 0.874, ranking 6th in the competition.
  • Task 2 (Multiclass Classification): Achieved a macro-averaged F1 score of 0.628, securing 1st position in the competition.

This performance significantly surpassed traditional ML models and other DL architectures, including other transformer-based approaches. For instance, in Task 1, their F1 score was higher than mBERT's 0.862. In Task 2, their 0.628 F1 score was substantially better than ML models, which struggled to reach even 0.3.

Limitations and Future Work

The authors acknowledge two primary limitations:

  • Dataset Size and Imbalance: The multiclass dataset is considered insufficient and imbalanced, which hinders the model's ability to distinguish nuanced categories.
  • Subtle Misinformation: The model struggles with effectively classifying subtle forms of news content, particularly those blending partial facts with deceptive elements.

For future work, they suggest:

  • Expanding Datasets: Creating larger and more balanced datasets for Malayalam fake news detection.
  • Exploring Advanced Architectures: Investigating more sophisticated models, potentially including Large Language Models (LLMs), to improve performance and better capture nuanced distinctions.
  • Focusing on Subtle Content: Developing methods specifically designed to detect more sophisticated and subtly deceptive misinformation.

Summary updated

BibTeX
@inproceedings{haq2025kcrl,
  author = {Fariha Haq and Md. Tanvir Ahammed Shawon and Md Ayon Mia and Golam Sarwar Md. Mursalin and Muhammad Ibrahim Khan},
  title = {{KCRL@DravidianLangTech 2025: Multi-Pooling Feature Fusion with XLM-RoBERTa for Malayalam Fake News Detection and Classification}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure