KEC_TECH_TITANS@DravidianLangTech 2025:Sentiment Analysis for Low-Resource Languages: Insights from Tamil and Tulu using Deep Learning and Machine Learning Models

Malliga Subramanian, Kogilavani Shanmugavadivel, Dharshini S, Deepiga P, Praveenkumar C, Ananthakumar SNAACL 2025

TL;DR

This paper tackles the challenge of sentiment analysis for Tamil and Tulu, two low-resource Dravidian languages, by comparing traditional machine learning models with advanced deep learning approaches like BERT and mBERT. The study reveals that transformer-based models significantly outperform traditional methods, highlighting their potential for improving sentiment analysis in under-resourced languages, even in the presence of code-mixing.

What Problem Does This Paper Address?

Sentiment analysis, the task of determining the emotional tone of text, is crucial for understanding public opinion, monitoring social media, and analyzing customer feedback. While significant progress has been made for high-resource languages like English, low-resource languages, particularly those in the Dravidian language family such as Tamil and Tulu, present unique challenges. These include linguistic diversity, complex grammar, limited availability of annotated datasets, and the prevalent phenomenon of code-mixing (users switching between languages, e.g., Tamil, Tulu, and English, within a single sentence). This study aims to bridge this gap by evaluating various machine and deep learning models for sentiment classification in Tamil and Tulu, with a specific focus on their effectiveness in handling these complexities.

Methods

The researchers explored a range of machine learning and deep learning techniques. Traditional methods included:

  • Support Vector Machines (SVM): A powerful algorithm for classification tasks.
  • K-Nearest Neighbors (KNN): A simple yet effective instance-based learning algorithm.
  • Decision Trees (DT): Used for exploring rule-based classification.
  • Logistic Regression: Employed for binary classification tasks.

On the deep learning front, they investigated:

  • Convolutional Neural Networks (CNNs): Effective at capturing local patterns in text.
  • Recurrent Neural Networks (RNNs) and Gated Recurrent Units (GRUs): Capable of handling sequential dependencies in text.
  • Transformer-based models: Specifically BERT (Bidirectional Encoder Representations from Transformers) and its multilingual variant mBERT. These models are known for their ability to learn contextual embeddings and capture long-range dependencies, making them state-of-the-art for many NLP tasks.

The study also incorporated preprocessing steps like text normalization, tokenization, stopword removal, lemmatization/stemming, and feature extraction using techniques like Bag-of-Words (BoW), TF-IDF, and word embeddings.

Datasets

The study utilized a custom dataset comprising Tamil-Tulu code-mixed text collected from social media platforms like Twitter and Facebook, along with regional online forums. The dataset consisted of 8,000 training samples and 2,000 validation samples.

  • Tamil Dataset: Showed a distribution of Positive, Negative, and Mixed Feelings sentiments, with a portion categorized as "Unknown State" which was filtered out.
  • Tulu Dataset: Included a significant number of "Not Tulu" samples that were excluded. The actual sentiment labels comprised Positive, Neutral, Mixed, and Negative sentiments.

The dataset exhibited class imbalance, particularly in the Mixed category, which was addressed using oversampling techniques like SMOTE and random resampling.

Key Insights and Ideas

  • Transformer Dominance: Transformer-based models, especially mBERT, significantly outperform traditional machine learning models for sentiment analysis in Tamil and Tulu. This is attributed to their superior ability to understand contextual nuances, which is critical for code-mixed text.
  • Code-Mixing is a Major Hurdle: The presence of code-mixing between Tamil, Tulu, and English poses a significant challenge. While mBERT showed promise, it still encountered occasional errors, indicating that even pre-trained multilingual models may need further fine-tuning on specific code-mixed datasets.
  • Data Scarcity is Pervasive: The lack of large, annotated datasets is a persistent bottleneck for low-resource languages like Tulu. This limitation directly impacts model accuracy and robustness.
  • Transfer Learning Potential: The success of mBERT highlights the power of transfer learning, where models pre-trained on vast multilingual corpora can be adapted to specific low-resource languages, providing a strong foundation for sentiment analysis.

Key Results

The study evaluated models using metrics such as accuracy, precision, recall, and F1-score.

  • BERT achieved the highest accuracy at 85%, followed by SVM at 83%.
  • mBERT also demonstrated strong performance, outperforming traditional models.
  • Traditional models like KNN and RNN showed lower performance compared to BERT and SVM.
  • The confusion matrix and AUC-ROC curve provided further insights into classification errors and the models' ability to distinguish between sentiment classes.

Limitations and Future Work

The authors acknowledge several limitations:

  • Dataset Imbalance: Despite oversampling, class imbalance, especially in the "Mixed Feelings" category, still affects performance.
  • Computational Cost: Deep learning models like BERT and mBERT are computationally expensive, making real-time sentiment analysis challenging.
  • Code-Mixing Nuances: While mBERT performed well, it was not specifically trained for Tamil-Tulu mixing, leading to occasional errors.
  • Limited Annotated Data: The scarcity of labeled data continues to impact accuracy.

Future work suggested includes:

  • Developing advanced data augmentation techniques to further improve class balance.
  • Exploring lighter transformer models or compression techniques for real-time applications.
  • Fine-tuning transformer models specifically on Tamil-Tulu code-mixed datasets.
  • Expanding the dataset with more labeled examples and incorporating domain-specific lexicons.

Summary updated

BibTeX
@inproceedings{subramanian2025kec,
  author = {Malliga Subramanian and Kogilavani Shanmugavadivel and Dharshini S and Deepiga P and Praveenkumar C and Ananthakumar S},
  title = {{KEC\_TECH\_TITANS@DravidianLangTech 2025:Sentiment Analysis for Low-Resource Languages: Insights from Tamil and Tulu using Deep Learning and Machine Learning Models}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure