UGSD: User Generated Sentiment Dictionaries from Online Customer Reviews

Chun-Hsiang Wang, Kang-Chun Fan, Chuan-Ju Wang, Ming-Feng TsaiAAAI 2019 · DOI

Abstract

Sentiment analysis is a critical task in natural language processing, yet its effectiveness is often limited by the domain-specific nature of sentiment-bearing words. Traditional sentiment dictionaries are either generic, failing to capture context-specific nuances, or require extensive manual annotation. This paper presents UGSD (User Generated Sentiment Dictionaries), a novel framework that automatically constructs high-quality sentiment lexicons by mining the relationship between review text and numerical ratings. By treating ratings as special tokens within a shared embedding space, UGSD learns to associate words with specific sentiment levels (from 1 to 5 stars) across diverse domains such as restaurants, attractions, and electronics.

Introduction

The core challenge in sentiment analysis is the domain dependency of language. For instance, the word "soft" acts as a positive descriptor for a mattress but a negative one for a laptop case. Generic dictionaries often miss these distinctions. Furthermore, existing methods for creating domain-specific dictionaries often rely on "seed words" or external linguistic resources like WordNet, which may not be available for all languages or specialized domains.

UGSD addresses these issues by proposing a "data-driven" approach. It utilizes the raw text and star ratings already present on major review platforms like Yelp, Amazon, and TripAdvisor. This allows for the creation of dictionaries that are inherently tailored to the specific vocabulary and sentiment expressions of a given domain without requiring manual intervention.

Methodology

The UGSD framework follows a structured four-stage process to transform raw review data into a functional sentiment lexicon.

1. Candidate Sentiment Word Selection

The system first applies Part-of-Speech (POS) tagging to identify potential sentiment-bearing words, primarily adjectives and adverbs. A key feature of UGSD is its handling of linguistic modifiers; it uses heuristics to create compound tokens for negations and intensifiers (e.g., "not_good" or "very_happy"). Additionally, Named Entity Recognition (NER) is employed to filter out proper nouns, ensuring the dictionary remains focused on descriptive sentiment.

2. Review Transformation and Representation Learning

The central innovation of UGSD lies in how it treats numerical ratings. The authors replace the name of the entity being reviewed with a symbol representing the user's rating (e.g., R1 for one star, R5 for five stars). This transformed text is then processed by a word embedding model that preserves co-occurrence proximity.

Because rating symbols now appear in the same context as descriptive words, the model learns a geometric space where "positive" words are located near R5 and "negative" words near R1. This effectively turns a supervised labeling problem into an unsupervised manifold learning task, where ratings act as semantic anchors.

3. Lexicon Construction

Finally, the framework maps each candidate word to its most relevant rating category. The authors utilize two schemes: Maximum Cosine Similarity and a Z-score scheme. This results in a fine-grained dictionary capable of distinguishing between various levels of sentiment intensity, rather than just binary positive or negative labels.

Experimental Results

The authors evaluated UGSD on three large-scale datasets: Yelp Restaurant Reviews (186k), TripAdvisor Attraction Reviews (2.8M), and Amazon Product Reviews (123k).

Sentiment Classification Performance

UGSD was tested against several baselines, including the Stanford Yelp Dictionary and generic resources like SentiWordNet. The results indicated that UGSD dictionaries consistently achieved higher F1-scores. Remarkably, UGSD dictionaries containing only a few hundred words often outperformed generic dictionaries with tens of thousands of entries, highlighting the efficiency of domain-specific learning.

Performance of positive and negative sentiment dictionaries across different thresholds.

Entity Ranking

The dictionaries were also applied to the task of ranking entities (e.g., attractions or products) based on the sentiment expressed in their reviews. Measured by Normalized Discounted Cumulative Gain (NDCG), UGSD significantly outperformed frequency-based methods and other sentiment-based baselines across all tested datasets.

Entity ranking performance (NDCG) on TripAdvisor and Amazon datasets.

Key Insights and Limitations

UGSD demonstrates that ratings can serve as powerful anchors for organizing semantic space. By moving beyond binary sentiment, the framework allows for more nuanced analysis. However, some limitations remain. Currently, UGSD assigns a review's overall rating to all sentiment words within it, which may be problematic for reviews mentioning multiple aspects (e.g., "great food but slow service"). Future research into aspect-based sentiment integration could further refine these dictionaries. Additionally, as a representation learning approach, UGSD requires a substantial volume of data to produce stable embeddings, which may limit its use in very niche domains.

Conclusion

UGSD offers a scalable and effective solution for generating domain-specific sentiment dictionaries. By leveraging the inherent structure of online reviews, it provides a path toward more accurate and context-aware sentiment analysis without the need for manual labeling.

Summary updated

BibTeX
@inproceedings{wang2019ugsd,
  author = {Chun-Hsiang Wang and Kang-Chun Fan and Chuan-Ju Wang and Ming-Feng Tsai},
  title = {{UGSD: User Generated Sentiment Dictionaries from Online Customer Reviews}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.1609/aaai.v33i01.3301313},
}
Figure