Towards Enhancing Lexical Resource and Using Sense-annotations of OntoSenseNet for Sentiment Analysis

Sreekavitha Parupalli, Vijjini Anvesh Rao, Radhika MamidiCOLING 2018

TL;DR

This paper introduces a crowd-sourcing tool to enrich OntoSenseNet, a Telugu lexical resource, with sense-annotations. They demonstrate that incorporating these sense-based features, alongside word-level sentiment polarity, significantly improves sentiment analysis performance for Telugu text, particularly with classifiers like Neural Networks.

What Problem Does This Paper Address?

Sentiment analysis, the task of determining the emotional tone of text, is crucial for understanding public opinion. While word-level sentiment analysis has seen progress, accurately capturing nuanced meaning often requires understanding the specific sense of a word, especially in languages like Telugu which have rich lexical resources but are less explored than English. Existing sentiment analysis approaches often treat words in isolation or rely on general sentiment lexicons. This paper addresses the gap of leveraging fine-grained semantic information (word senses) from a specialized lexical resource (OntoSenseNet) to enhance sentiment analysis for Telugu.

Methods

The core of this work involves developing a crowd-sourcing platform named "Parupalli Padajaalam" to populate OntoSenseNet with sentiment polarity and ontological sense annotations. The platform allows users to add words, their meanings, and sample sentences. For sentiment analysis experiments, the authors employ a benchmark Word2Vec approach that averages word vectors from a review. They then augment this baseline by incorporating two types of features derived from OntoSenseNet:

  1. Word-level Polarity Features: These are standard features derived from existing sentiment lexicons like SentiWordNet, including counts of positive/negative unigrams and bigrams.
  2. OntoSenseNet Features: This is the novel contribution. They extract features based on:
  3. Verb Sense-types: Frequency of 7 predefined sense-types for verbs (e.g., "To Know," "To Move").
  4. Adverb Sense-classes: Frequency of 4 sense-classes for adverbs (e.g., "Spatial," "Temporal").

These additional features are integrated into the review vectors, and their impact is evaluated using various classifiers like Linear SVM, Gaussian SVM, Random Forest, Neural Network, and K-Nearest Neighbor.

Datasets

  • OntoSenseNet (Telugu): A lexical resource developed for the Telugu language, containing sense-type annotations for verbs and sense-classes for adverbs. The paper describes efforts to enrich this resource through crowd-sourcing.
  • Sentiraama Corpus: A Telugu corpus used for sentiment analysis, containing reviews from various domains (movie reviews, song lyrics, product reviews, book reviews). This corpus is used to construct the review vectors for experiments.
  • Telugu Wikipedia: Used for extracting Verb-Adverb and Adverb-Verb pairs to analyze adverbial class distribution.
  • SentiWordNet: A lexical resource for English with sentiment polarity annotations, used for baseline word-level polarity features.

Key Insights and Ideas

  • Sense matters for sentiment: The paper strongly suggests that understanding the specific sense of a word is crucial for accurate sentiment analysis, especially in a morphologically rich language like Telugu.
  • Crowd-sourcing for resource enrichment: The developed platform demonstrates a practical approach to scaling the annotation of lexical resources, which is often a bottleneck.
  • OntoSenseNet as a valuable feature source: The OntoSenseNet features, particularly verb sense-types and adverb sense-classes, provide semantic context that goes beyond simple polarity, leading to improved sentiment classification.
  • Classifier sensitivity to new features: Different classifiers react differently to the new features. K-Nearest Neighbor shows a performance drop, possibly due to its inability to weigh features, while Random Forest and Neural Networks benefit from the added semantic information.

Key Results

  • The inclusion of OntoSenseNet features, combined with word-level polarity features, led to improved sentiment analysis accuracy for most classifiers.
  • Neural Network Performance: The Neural Network classifier showed the most promising results, achieving an accuracy of 83.08% with OntoSenseNet features alone and 81.09% with both OntoSenseNet and word-level polarity features, compared to 81.09% with just Word2Vec.
  • Precision, Recall, and F1-scores for Neural Network: With OntoSenseNet features, the Neural Network achieved a precision of 0.833, recall of 0.811, and F1-score of 0.829, showing improvements over the baseline.
  • Adverbial Class Distribution: Analysis of Telugu Wikipedia revealed a higher proportion of Adverb-Verb pairs compared to Verb-Adverb pairs, aligning with Telugu language structure.

Limitations and Future Work

  • Limited Annotation Coverage: While the crowd-sourcing tool aims to address this, OntoSenseNet still has a significant number of adjectives and verbs that require annotation.
  • Classifier Performance Variance: The paper notes that Neural Networks showed high variance in repeated trials, suggesting further tuning or investigation might be needed.
  • Future Work: The authors plan to continue annotating the remaining adjectives and verbs in OntoSenseNet. They also aim to annotate bigram (verb, adverb) pairs and further explore the insights from adverbial class distribution for Word Sense Disambiguation (WSD) tasks. The ultimate goal is to leverage these enriched lexical resources for improved sentiment analysis.

Summary updated

BibTeX
@inproceedings{parupalli2018towards,
  author = {Sreekavitha Parupalli and Vijjini Anvesh Rao and Radhika Mamidi},
  title = {{Towards Enhancing Lexical Resource and Using Sense-annotations of OntoSenseNet for Sentiment Analysis}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2018},
}
Figure