Sentiment Analysis on Code-Switched Dravidian Languages with Kernel Based Extreme Learning Machines

Mithun Kumar S R, Lov Kumar, Aruna MalapatiACL 2022

TL;DR

This paper tackles the challenging task of sentiment analysis on code-switched Dravidian languages mixed with English, a common phenomenon in online communication. They propose using Kernel-based Extreme Learning Machines (ELMs) as a faster and effective alternative to deep learning models, finding that polynomial kernels and specific data balancing techniques yield the best results.

What Problem Does This Paper Address?

Code-switching, the practice of alternating between two or more languages within a single conversation or text, is prevalent in many multilingual communities. While sentiment analysis on single languages is well-studied, applying it to code-switched text, especially involving less-resourced languages like Dravidian languages (Tamil, Kannada, Malayalam) mixed with English, presents significant challenges. The irregular sentence structures and vocabulary mixing make it difficult for traditional NLP models to accurately capture sentiment. Existing research has explored deep learning models, but these often require extensive training time. This paper aims to bridge this gap by investigating the effectiveness and efficiency of Kernel-based Extreme Learning Machines (ELMs) for this specific, complex task.

Methods

The authors employ a multi-staged pipeline for their experiments.

  • Data Preprocessing: Standard steps like case conversion, stopword removal, and lemmatization were applied. Crucially, they filtered out data labeled as "Not in the target language."
  • Word Embedding: They utilized pre-trained FastText word vectors in 300 dimensions to represent sentences, capturing semantic meaning.
  • Feature Selection: Two approaches were used: retaining all features from the word embeddings ("All") and applying Principal Component Analysis (PCA) for dimensionality reduction.
  • Data Balancing: To address the imbalanced nature of their dataset (skewed towards positive sentiments), they experimented with both the imbalanced data and an oversampling technique called SMOTE (Synthetic Minority Over-sampling Technique).
  • Kernel-based ELMs: The core of their approach involves Extreme Learning Machines (ELMs) with different kernel functions: Radial Basis Functions (RBF), Linear (LIN), and Polynomial (POLY). ELMs are known for their fast training times due to their single-layer feedforward network structure with randomly initialized hidden layer weights.

Datasets

The study utilized three publicly available, manually labeled, code-mixed benchmark datasets for Dravidian languages:

  • Kannada code-switched corpus (Hande et al., 2020)
  • Tamil code-switched corpus (Chakravarthi et al., 2020b)
  • Malayalam code-switched corpus (Chakravarthi et al., 2020a)

The dataset was multi-class, with labels including 'Positive', 'Negative', 'Mixed Feelings', and 'Unknown State'. A split of the data across these classes for each language is presented in Table 1.

Key Insights and Ideas

  • ELMs as a Viable Alternative: The research demonstrates that ELMs can be a strong contender for sentiment analysis on code-switched Dravidian languages, offering a significant speed advantage over deep learning models without sacrificing performance.
  • Polynomial Kernels Shine: Contrary to expectations that linear kernels might suffice for text, the study found that non-linear kernels, particularly Polynomial kernels, consistently outperformed Linear and RBF kernels, especially when dealing with high-dimensional word embeddings.
  • Feature Selection Matters: The analysis suggests that retaining all features from the word embeddings generally leads to better accuracy compared to dimensionality reduction techniques like PCA. This indicates that the rich spatial information in the higher dimensions of word vectors is crucial for capturing nuanced sentiment in code-switched text.
  • SMOTE's Mixed Impact: While SMOTE was hypothesized to improve performance, its effect was nuanced. It showed promise in improving AUC (Area Under the ROC Curve) when used with all features, but it sometimes worsened accuracy, potentially due to over-generalization or augmenting noisy regions.

Key Results

  • Speed: ELM training times were around 60 minutes for most languages, significantly faster than deep learning models.
  • Accuracy: The best accuracy was achieved when all features were selected, and Polynomial kernels were used. For instance, a median AUC of 0.79 was achieved with a polynomial kernel.
  • Kernel Performance: Polynomial kernels generally outperformed Linear and RBF kernels across various metrics.
  • Data Balancing: SMOTE techniques, when combined with all features, favored better AUC in ELM models.

Limitations and Future Work

  • Dimensionality Reduction: The paper notes that PCA negatively impacted accuracy, suggesting that higher dimensions in word embeddings are beneficial for capturing context in code-switched text.
  • SMOTE's Impact: While SMOTE improved AUC in some cases, its effect on accuracy was not consistently positive, indicating potential issues with over-generalization.
  • Future Work: The authors suggest future research should focus on improving word embedding techniques and language identification specifically for code-switched data within the context of kernel-based ELMs. They also mention that ELM models performed better in chosen metrics compared to traditional ensemble classifiers.

Summary updated

BibTeX
@inproceedings{r2022sentiment,
  author = {Mithun Kumar S R and Lov Kumar and Aruna Malapati},
  title = {{Sentiment Analysis on Code-Switched Dravidian Languages with Kernel Based Extreme Learning Machines}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
}
Figure