Advancing Sentiment Analysis in Tamil-English Code-Mixed Texts: Challenges and Transformer-Based Solutions

Mikhail Krasitskii, Olga Kolesnikova, Liliana Chanona Hernandez, Grigori Sidorov, Alexander GelbukhNAACL 2025

TL;DR

This paper tackles the complex task of sentiment analysis in Tamil-English code-mixed text, a common phenomenon in digital communication. The authors evaluate several state-of-the-art transformer models, finding that RemBERT and mT5 offer the best performance, while also highlighting key challenges like linguistic variability and data scarcity that require further research.

What Problem Does This Paper Address?

Code-mixing, the practice of blending two or more languages within a single utterance or text, is increasingly prevalent in online communication. This paper focuses on Tamil-English code-mixed texts, which present unique challenges for sentiment analysis. These include:

  • Grammatical Inconsistencies: Mixing sentence structures from both languages.
  • Orthographic Variations: The simultaneous use of the Tamil script and its transliteration into the Latin alphabet, leading to tokenization difficulties.
  • Phonetic Ambiguities: Non-standard transliterations and spelling variations for the same word, complicating text normalization.
  • Data Scarcity: A lack of large, diverse, and well-annotated datasets for training robust models.

Traditional NLP methods struggle with these complexities, necessitating the exploration of advanced transformer-based models.

Methods

The study evaluates the performance of several transformer-based models for sentiment analysis on Tamil-English code-mixed texts. The selected models are:

  • XLM-ROBERTa: A multilingual model pretrained on data from 100 languages, suitable for cross-lingual tasks.
  • mT5: A multilingual variant of the T5 model, capable of generative tasks and processing code-mixed text.
  • IndicBERT: A model specifically designed for Indian languages, utilizing subword tokenization to capture morphological features.
  • RemBERT: An architecture designed for multitask learning with enhanced embeddings for complex linguistic structures.

A multi-step data preprocessing pipeline was implemented, including:

  • Text Normalization: Standardizing data formats and unifying transliterated variations.
  • Tokenization: Using SentencePiece for subword segmentation to better handle phonetic variations.
  • Data Augmentation: Employing back-translation to generate additional training examples and increase data diversity.

Datasets

The paper utilizes existing datasets for training and evaluation:

  • Sentiment Analysis for Indian Languages (SAIL): Contains 15,000 annotated tweets in both Tamil script and Romanized versions, labeled as positive, neutral, or negative.
  • CodeMixed Data Repository (CMD-Tamil): A dataset of 10,000 sentences in the Latin script, designed for easier processing by transformer models.
  • DravidianCodeMix: A dataset with 12,000 examples of code-mixed texts for Dravidian languages, including Tamil.

The authors note that while these datasets provide a foundation, their limited size and thematic diversity pose significant challenges for machine learning model generalization.

Key Insights and Ideas

  • Transformer models show promise: State-of-the-art transformer architectures are significantly better at handling the complexities of code-mixed text compared to traditional methods.
  • RemBERT and mT5 lead the pack: These models demonstrated superior performance in sentiment classification for Tamil-English code-mixed text.
  • Linguistic factors are critical: The use of the Latin script generally increases accuracy due to better adaptation of pre-trained embeddings. Conversely, phonetic errors and complex grammatical structures significantly reduce model performance.
  • Sarcasm and idioms are major hurdles: The highest error rates are associated with the misinterpretation of sarcasm and idiomatic expressions, highlighting a need for specialized handling of these linguistic nuances.
  • Data limitations persist: Despite efforts, the available datasets are still insufficient in size and diversity, hindering model generalization and robust performance.

Key Results

The study evaluated models using Accuracy, Precision, Recall, F1-score, and BLEU-score.

  • RemBERT achieved the highest accuracy (87.5%) and F1-score (86.4%).
  • mT5 also performed strongly with an accuracy of 86.3% and an F1-score of 85.1%.
  • IndicBERT and XLM-ROBERTa showed slightly lower results, potentially due to less adaptation to the Tamil language.
  • Error Analysis: Sarcasm and idioms accounted for 35% of errors, followed by complex syntax (28%), phonetic errors (22%), and script differences (15%).

Limitations and Future Work

The authors acknowledge several limitations:

  • Limited Data: Existing datasets are insufficient in size and thematic diversity, restricting model generalization.
  • Low-Resource Constraints: Even advanced models show reduced performance when processing Tamil due to its low-resource status.
  • Sarcasm Recognition: Current models lack built-in mechanisms to correctly interpret sarcasm and complex linguistic constructs.
  • Alphabet Mixing: The switch between Tamil and Latin scripts complicates tokenization.

Future research directions proposed include:

  • Expanding and annotating datasets: Creating larger, more balanced datasets with greater linguistic and thematic diversity.
  • Developing phonetics-aware models: Integrating linguistic rules and normalization modules to correct spelling and phonetic errors.
  • Using hybrid analysis methods: Combining transformer models with traditional NLP approaches.
  • Applying semi-supervised and unsupervised methods: Leveraging active learning and self-supervision to reduce reliance on annotated data.
  • Optimizing models for code-mixing: Designing specialized architectures that account for code-switching and contextual dependencies.
  • Training models with cultural context: Incorporating socio-linguistic factors, including regional slang and idiomatic expressions.

Summary updated

BibTeX
@inproceedings{krasitskii2025advancing,
  author = {Mikhail Krasitskii and Olga Kolesnikova and Liliana Chanona Hernandez and Grigori Sidorov and Alexander Gelbukh},
  title = {{Advancing Sentiment Analysis in Tamil-English Code-Mixed Texts: Challenges and Transformer-Based Solutions}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure