MUCS@DravidianLangTech@ACL2022: Ensemble of Logistic Regression Penalties to Identify Emotions in Tamil Text

Asha Hegde, Sharal Coelho, Hosahalli ShashirekhaACL 2022

TL;DR

This paper tackles the challenge of automatically identifying emotions in Tamil text, a crucial task for understanding public sentiment and opinion. The authors propose an ensemble of Logistic Regression classifiers, leveraging different regularization penalties (L1, L2, and Elasticnet) and character n-grams, to achieve this. While achieving a respectable 4th place in the shared task, the results highlight the difficulties posed by imbalanced datasets in this domain.

What Problem Does This Paper Address?

Emotion Analysis (EA) is the process of automatically identifying and categorizing emotions expressed in text. In recent years, social media has become a primary platform for users to share their feelings, opinions, and reactions to various topics. This wealth of user-generated content offers valuable insights for businesses, governments, and researchers. However, manually analyzing this vast amount of data is impractical and error-prone. Furthermore, the unique characteristics of social media text, such as informal language and abbreviations, make EA a challenging task. While most EA research has focused on English, there's a significant gap in developing effective tools for low-resource languages like Tamil. This paper aims to address this gap by developing an EA model specifically for Tamil social media comments.

Methods

The core of the proposed methodology is an Ensemble of Logistic Regression (LR) classifiers. This approach combines multiple LR models, each trained with a different regularization penalty:

  • L1 Regularization (LASSO): This penalty adds the absolute magnitude of coefficients to the loss function, encouraging sparsity and effectively performing feature selection by shrinking some coefficients to zero.
  • L2 Regularization (Ridge): This penalty adds the squared magnitude of coefficients to the loss function, shrinking coefficients towards zero but rarely making them exactly zero.
  • Elasticnet Regularization: This method combines both L1 and L2 penalties, aiming to leverage the feature elimination capabilities of L1 and the coefficient shrinkage of L2 for improved predictions.

The models were trained using TF-IDF (Term Frequency-Inverse Document Frequency) vectorized character bigrams and trigrams extracted from the Tamil text. This approach captures local word order and patterns within words, which can be particularly useful for languages with rich morphology. For the final classification, a soft voting mechanism was employed to combine the predictions of the individual LR classifiers.

Datasets

The paper utilizes a dataset specifically curated for the "Emotion Analysis in Tamil" shared task at DravidianLangTech@ACL 2022. This dataset contains Tamil comments labeled with one of eleven emotion categories. The authors note that the dataset exhibits an imbalance in class distribution, with the 'Neutral' class being the most frequent and 'Fear' being the least frequent.

Key Insights and Ideas

  • Ensemble of Regularized Classifiers: Combining LR models with different regularization penalties (L1, L2, Elasticnet) is a robust strategy for capturing diverse patterns and mitigating overfitting, especially in complex text data.
  • Character N-grams for Tamil: Using character bigrams and trigrams proved effective for feature extraction in Tamil, likely due to the language's morphological richness and the ability of character-level features to handle variations and misspellings.
  • Impact of Data Imbalance: The significant class imbalance in the dataset was identified as a major factor contributing to the low F1-scores, particularly for minority classes. This highlights a common challenge in real-world NLP tasks and the need for effective strategies to address it.
  • Shared Task Context: The paper demonstrates the application of established NLP techniques within the specific context of a shared task, providing a benchmark for future research in Tamil emotion analysis.

Key Results

The proposed Ensemble of Logistic Regression classifiers achieved a macro-averaged F1-score of 0.04 on the test set, securing 4th place in the "Emotion Analysis in Tamil" shared task. The model achieved a macro-averaged F1-score of 0.20 on the development set. The authors attribute the low scores to the imbalanced nature of the dataset, where the 'Neutral' class constituted 34.07% of the data, while 'Fear' represented only 0.70%.

Limitations and Future Work

Limitations:

  • Dataset Imbalance: The most significant limitation acknowledged is the imbalanced distribution of emotion classes in the dataset, which negatively impacted the model's performance, particularly for underrepresented emotions.

Future Work:

  • The authors intend to investigate different sets of features and explore various re-sampling methods (e.g., oversampling, undersampling) to address the class imbalance issue. This is a crucial next step for improving the model's overall effectiveness.

Summary updated

BibTeX
@inproceedings{hegde2022mucs,
  author = {Asha Hegde and Sharal Coelho and Hosahalli Shashirekha},
  title = {{MUCS@DravidianLangTech@ACL2022: Ensemble of Logistic Regression Penalties to Identify Emotions in Tamil Text}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
}
Figure