Identifying Emotional and Polar Concepts via Synset Translation

Logan Woudstra, Moyo Dawodu, Frances Igwe, Senyu Li, Ning Shi, Bradley Hauer, Grzegorz KondrakNAACL 2024

TL;DR

This paper introduces SentiSynset, a novel, large-scale resource for identifying emotional and polar concepts at the synset (word sense) level. By leveraging multilingual translations and existing word-level sentiment lexicons, they achieve high precision in labeling synsets, overcoming limitations of previous, smaller, and less comprehensive resources.

What Problem Does This Paper Address?

Sentiment analysis, encompassing emotion identification and polarity classification, aims to understand the emotional tone and sentiment expressed in text. While word-level sentiment lexicons exist, they struggle with polysemy – words having multiple meanings, each potentially carrying a different sentiment. Existing synset-level resources are often limited in size and coverage, built through semi-automatic methods that require manual annotation of a core set of synsets. This paper tackles the challenge of creating a comprehensive and high-precision synset-level sentiment resource that can be automatically generated and applied across languages.

Methods

The core of the proposed method is a translation-based approach. The authors hypothesize that different senses of a polysemous word might translate differently, allowing for disambiguation. They leverage multilingual word-level sentiment lexicons (specifically, translated versions of EmoLex) to identify the sentiment of translated lemmas.

The process involves:

  1. Identifying a "core" set of synsets: For each synset, they find its lemmas across multiple languages. If a lemma in a foreign language has a sentiment label that aligns with an English lemma in the same synset, it contributes to the synset's sentiment. The synset is then labeled with the sentiment that appears most frequently across its translations.
  2. Extending the core: They use WordNet's graph structure and sentiment-preserving relations (like antonymy, similarity, etc.) to propagate sentiment labels from the core synsets to their neighbors. This extension is done carefully to maintain high precision, avoiding recursive or transitive propagation.

Datasets

  • EmoLex (NRC Word-Emotion Association Lexicon): A word-level sentiment lexicon containing emotional and polar labels for English words. Crucially, EmoLex has been translated into 108 languages, forming the basis for the multilingual sentiment lexicons used in this work.
  • WordNet: The Princeton WordNet, a large lexical database of English, is used as the target resource for synset labeling.
  • Validation and Test Sets: Two disjoint sets, each containing 1,000 synsets, were created for evaluation. These sets include both sentimental instances (from SentiSense) and non-sentimental instances (from EmoLex and LIWC dictionary).
  • BabelNet and Google Translate: Used as sources for obtaining multilingual translations of WordNet synsets.

Key Insights and Ideas

  • Multilinguality as a Disambiguation Tool: The core idea that translating polysemous words into different languages can help resolve their ambiguous meanings is a powerful insight. Different senses of a word might have distinct translations, and by examining the sentiment of these translations, one can infer the sentiment of the original word sense.
  • High Precision over Coverage: The authors prioritize creating a highly accurate resource, even if it means not labeling every single synset. This is achieved by setting a high confidence threshold for including synsets in the "core" set.
  • Scalability through Automation: By automating the process of core identification and extension, the method can generate a significantly larger sentiment resource (SentiSynset) compared to previous semi-automatic approaches.
  • Cultural Nuances in Sentiment: The error analysis highlights that while sentiment might be largely universal, cultural differences can influence how certain concepts are perceived, leading to disagreements even among native speakers.

Key Results

  • SentiSynset Resource: The paper introduces SentiSynset, containing 12,429 emotional synsets and 15,567 polar synsets, significantly larger than prior resources.
  • High Precision and Recall:
    • Emotion Identification: Achieved 96.0% precision and 57.2% recall on the test set.
  • Polarity Classification: Achieved 92.0% precision and 67.0% recall on the test set.
  • Performance Comparison: The proposed method outperforms other approaches, including random EmoLex baselines, sentence embeddings, SentiWordNet, and ChatGPT, in both emotion identification and polarity classification tasks.
  • Polysemy Resolution: The method demonstrates high accuracy (94.4%) in correctly classifying polysemous words with opposite sentiments.

Limitations and Future Work

  • Parallel Polysemy: The method struggles with concepts that exhibit parallel polysemy across many languages, where different senses of a word are consistently translated to the same terms in multiple languages, hindering disambiguation.
  • EmoLex Errors: Translation errors in the multilingual EmoLex lexicons can lead to misclassifications, particularly when the most frequent sense (MFS) of a word is non-sentimental but the lexicon incorrectly assigns sentiment.
  • Subjectivity and Cultural Differences: The inherent subjectivity of sentiment and cultural variations in perception can lead to errors, especially for concepts that are not universally perceived in the same way.
  • Future Work: The authors suggest that their method is not dependent on EmoLex and could leverage information from multiple word-level lexicons to further improve the quality and size of SentiSynset. They also note that the resource can be paired with word-sense disambiguation techniques for downstream sentiment analysis tasks.

Summary updated

BibTeX
@inproceedings{woudstra2024identifying,
  author = {Logan Woudstra and Moyo Dawodu and Frances Igwe and Senyu Li and Ning Shi and Bradley Hauer and Grzegorz Kondrak},
  title = {{Identifying Emotional and Polar Concepts via Synset Translation}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
}
Figure