KSK@DravidianLangTech 2025: Political Multiclass Sentiment Analysis of Tamil X (Twitter) Comments Using Incremental Learning

Kalaivani K S, Sanjay R, Thissyakkanna S M, Nirenjhanram S KNAACL 2025

TL;DR

This paper tackles the challenge of analyzing political sentiment in Tamil social media comments, which often feature a mix of Tamil and English (code-mixing). They propose using incremental learning with traditional machine learning models to adapt to the dynamic nature of social media language, achieving the best performance with an SGD Classifier.

What Problem Does This Paper Address?

The explosion of social media usage in India, particularly on platforms like X (formerly Twitter), has led to a significant increase in users communicating in their native languages, often mixed with English. This "code-mixing" presents a substantial challenge for sentiment analysis tools, as it deviates from standard linguistic rules and requires models that can handle linguistic diversity. Specifically, this paper focuses on classifying political comments into seven sentiment categories, aiming to monitor public opinion and ensure a safe online environment. Manual analysis is infeasible due to the sheer volume of data, and static models struggle to keep up with the ever-evolving language and trends on social media.

Methods

The researchers explored four machine learning models: Stochastic Gradient Descent (SGD) Classifier, Random Forest Classifier, Decision Tree Classifier, and Multinomial Naive Bayes Classifier. The core innovation lies in their application of incremental learning. Instead of retraining models from scratch with the entire dataset, they trained the models in smaller batches. This allows the models to learn progressively, adapt to new data and evolving language patterns without forgetting previously learned information, and maintain performance over time. This approach is particularly beneficial for real-time applications and large-scale datasets.

Preprocessing steps included:

  • Removal of Hashtags, URLs, and Mentions: These elements were removed as they often don't contribute to sentiment.
  • Whitespace Normalization: Extra spaces were cleaned up for consistency.
  • Tokenization: Text was split into individual words.

Datasets

The study utilized a dataset provided by DravidianLangTech for Social Media sentiment classification. This dataset contains code-mixed Tamil and English comments.

  • Training Dataset: 4533 rows with seven sentiment classes: negative, neutral, none of the above, opinionated, positive, sarcastic, and substantiated.
  • Validation Dataset: 544 rows.

The paper notes that the dataset contains seven distinct sentiment classes, which is a key aspect of their multiclass classification task.

Key Insights and Ideas

  • Incremental Learning is Crucial for Dynamic Social Media: The paper strongly advocates for incremental learning as a solution to the ever-changing nature of social media language and sentiment. It allows models to adapt without catastrophic forgetting, a significant advantage over traditional batch training.
  • SGD Classifier Excels in This Domain: Despite the complexity of code-mixed data, the SGD Classifier demonstrated the best performance, suggesting its robustness and efficiency for this specific task.
  • Code-Mixing Poses Unique Challenges: The study highlights how the blend of Tamil and English creates difficulties that require specialized approaches beyond standard sentiment analysis techniques.
  • Multiclass Sentiment is More Nuanced: Classifying into seven distinct sentiment categories provides a richer understanding of user opinions compared to binary sentiment analysis.

Key Results

The SGD Classifier achieved the highest training accuracy of 83.67% and a validation accuracy of 80.43%. This strong validation accuracy indicates good generalization to unseen data.

  • Random Forest Classifier: 82.56% training accuracy, 70.21% validation accuracy.
  • Logistic Regression: 80.82% training accuracy, 77.89% validation accuracy.
  • Naive Bayes Classifier: 78.78% training accuracy, 65.23% validation accuracy.

The paper notes that Naive Bayes struggled to generalize, likely due to its assumption of feature independence, which doesn't hold for code-mixed text. Random Forest showed a notable drop in validation accuracy, suggesting potential overfitting.

Limitations and Future Work

The authors acknowledge that while accuracy is important, it doesn't fully capture performance on imbalanced sentiment classes. They suggest future work could involve:

  • Exploring Deep Learning Models: Incorporating models like BERT or RoBERTa for potentially enhanced classification.
  • Weighted Loss Functions: To better handle imbalanced sentiment classes.
  • Data Augmentation Techniques: To increase the size and diversity of the training data.
  • Fine-tuning Pre-trained Models: Leveraging existing powerful models for code-mixed data.
  • Expanding the Dataset: To include more diverse political contexts.

Summary updated

BibTeX
@inproceedings{s2025ksk,
  author = {Kalaivani K S and Sanjay R and Thissyakkanna S M and Nirenjhanram S K},
  title = {{KSK@DravidianLangTech 2025: Political Multiclass Sentiment Analysis of Tamil X (Twitter) Comments Using Incremental Learning}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure