Team-Risers@DravidianLangTech 2025: AI-Generated Product Review Detection in Dravidian Languages Using Transformer-Based Embeddings

Sai Sathvik, Muralidhar Palli, Keerthana Nnl, Balasubramanian Palani, Jobin Jose, Siranjeevi RajamanickamNAACL 2025

TL;DR

This paper tackles the growing problem of AI-generated fake product reviews in low-resource Dravidian languages like Tamil and Malayalam. By leveraging transformer-based embeddings from Tamil-BERT and Malayalam-BERT, the authors achieve high accuracy in distinguishing between human and AI-written reviews, significantly improving upon traditional methods.

What Problem Does This Paper Address?

In today's digital marketplace, online reviews heavily influence consumer decisions. However, the rise of AI has enabled companies to generate fake reviews, artificially boosting their products' credibility and potentially misleading customers. While AI-generated text detection is an active research area, it's particularly challenging for low-resource languages like Tamil and Malayalam due to their complex linguistic structures and the limited availability of specialized NLP tools. This paper aims to bridge this gap by developing an effective method for detecting AI-generated product reviews in these Dravidian languages.

Methods

The proposed methodology involves a multi-stage process:

  • Text Preprocessing: This crucial step involves cleaning and standardizing the review text. It includes noise removal, sentence segmentation, and tokenization using language-specific tokenizers for Tamil and Malayalam. WordPiece tokenization is employed to handle morphologically rich words by breaking them down into subwords, preserving semantic relationships. Dynamic sequence length normalization ensures uniform input for the transformer models.
  • Embedding Layer: Instead of traditional methods like Bag of Words (BoW) or TF-IDF, the paper utilizes transformer-based embeddings. Specifically, it employs Tamil-BERT and Malayalam-BERT to generate dense, contextual representations of the text. These models leverage a multi-head self-attention mechanism to capture long-range dependencies within the text, which is vital for understanding nuanced language.
  • Classification: The contextual embeddings generated by the BERT models are then fed into a Feedforward Neural Network (FFN). This FFN uses GeLU activation in its hidden layers and a Softmax output layer to predict the probability of a review being AI-generated (class 0) or human-written (class 1).

Datasets

The study utilizes a dataset sourced from the DravidianLangTech-2025 shared task. The dataset comprises:

  • Tamil: 403 human-written reviews and 405 AI-generated reviews.
  • Malayalam: 400 human-written reviews and 400 AI-generated reviews.

The paper notes that AI-generated reviews tend to be shorter and have a lower word count distribution compared to human-written reviews, which exhibit a broader range of lengths.

Key Insights and Ideas

  • Transformer Embeddings Outperform Traditional Methods: The core insight is that transformer-based embeddings (Tamil-BERT, Malayalam-BERT) are significantly more effective than classical text encoding methods (BoW, TF-IDF) for detecting AI-generated reviews in Dravidian languages. This highlights the power of contextual understanding provided by these advanced models.
  • Language-Specific Models are Crucial: The success of using dedicated Tamil-BERT and Malayalam-BERT models underscores the importance of language-specific approaches for low-resource languages, rather than relying on generic multilingual models.
  • Preprocessing is Key for Low-Resource Languages: Effective preprocessing, including WordPiece tokenization, is vital for handling the morphological complexity of languages like Tamil and Malayalam, enabling transformer models to better capture linguistic nuances.

Key Results

The proposed FFN classifier with BERT embeddings achieved impressive results:

  • Tamil: 95.68% accuracy.
  • Malayalam: 88.75% accuracy.

These results significantly outperform traditional machine learning models (Decision Trees, Random Forests, SVM, Naive Bayes) when used with TF-IDF or BoW embeddings, as detailed in Table 2. For instance, the best-performing traditional model for Tamil (RF with TF-IDF) achieved 89.51% accuracy, while for Malayalam, NB with TF-IDF reached 80.00% accuracy.

Limitations and Future Work

The authors acknowledge several limitations:

  • Small Dataset Size: The relatively small size of the datasets used may impact the generalizability and robustness of the models.
  • Limited Representation of Offensive/Misogynistic Words: Low-resource languages often have less data for certain types of content, which can constrain the effectiveness of BERT models in detecting subtle forms of manipulation.

Future work directions include:

  • Expanding Datasets: Increasing the size and diversity of datasets is crucial for improving model performance.
  • Incorporating Multilingual Models: Exploring models like XLM-ROBERTa and IndicBert, which are trained on larger datasets and more tokens, could offer better performance, especially for mixed-code texts.
  • Enhancing Linguistic Diversity: Further research into capturing a wider range of linguistic variations within Dravidian languages is recommended.

Summary updated

BibTeX
@inproceedings{sathvik2025team,
  author = {Sai Sathvik and Muralidhar Palli and Keerthana NNL and Balasubramanian Palani and Jobin Jose and Siranjeevi Rajamanickam},
  title = {{Team-Risers@DravidianLangTech 2025: AI-Generated Product Review Detection in Dravidian Languages Using Transformer-Based Embeddings}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure