TL;DR
This paper introduces an unsupervised system for extracting aspects (e.g., "food," "service") and their associated sentiment from online reviews. Unlike previous methods that often rely on manual annotations or extensive knowledge bases, this approach uses a local topic model and automatically derived sentiment seed sets, making it more adaptable to new domains and languages.
What Problem Does This Paper Address?
The proliferation of online review sites has created a massive amount of user-generated text. Extracting actionable insights from this data is crucial for applications like product comparison, recommendation systems, and automated summarization. However, existing methods for identifying "aspects" (features or attributes of a product/service) and "sentiment" (positive, negative, or neutral opinions) often struggle with the diversity of products, the unstructured nature of reviews (including misspellings and slang), and the fact that sentiment can be highly context-dependent. Supervised methods are impractical due to the need for extensive labeled data for each new domain. While unsupervised methods exist, they often fail to capture fine-grained, rateable aspects or struggle with the interaction between aspects and sentiment.
Methods
The paper proposes a two-pronged unsupervised approach:
- Aspect Extraction: They employ a Local Latent Dirichlet Allocation (LDA) model. Unlike standard LDA which models topics across entire documents, their local LDA treats each sentence as a document and uses a small number of topics (10-20) to infer aspects. This sentence-level focus helps capture fine-grained, rateable aspects rather than broad, global topics. They use a cluster validation scheme to determine the optimal number of topics. Representative words for each aspect are identified using mutual information.
- Sentiment Detection: This component builds upon the extracted aspects.
- Adjective Extraction: They parse sentences to identify adjective-noun pairs, focusing on explicit conjunctions and negations.
- Graph Construction: A graph is built where adjectives are nodes, and edges represent their co-occurrence in modifying the same noun within a sentence. Crucially, they build separate graphs for each aspect to capture aspect-specific sentiment.
- Automatic Seed Set Generation: Instead of relying on manually curated lists of positive/negative words, they generate a seed set by identifying adjective pairs that differ only by common prefixes (e.g., "un-", "in-") or the negation marker "not-". The polarity is then propagated through the graph using a label propagation algorithm.
- Polarity Propagation: The algorithm assigns polarity scores to adjectives based on their connections in the aspect-specific graphs, ultimately determining the sentiment associated with each aspect.
Datasets
- Restaurant Reviews: The primary dataset is a publicly available corpus of over 50,000 restaurant reviews from Citysearch New York.
- Netbook Reviews: For domain independence testing, they collected 1086 reviews for four leading netbook computers from Amazon.com.
- Annotated Dataset: For evaluation, they used a subset of 3,400 sentences from the Citysearch corpus, manually labeled for aspect and sentiment by Ganu et al. (2009). This dataset provided six predefined aspect labels and sentiment labels (Positive, Negative, Neutral, Conflict).
- Seed Set: They also utilized a manually provided seed set of 128 positive and 88 negative adjectives from Fahrni and Klenner (2008) for comparison.
Key Insights and Ideas
- Sentence-Level Topic Modeling for Aspects: The insight that treating sentences as documents in LDA can reveal finer-grained, rateable aspects is key. This contrasts with document-level LDA, which tends to find broader themes.
- Aspect-Specific Sentiment Graphs: Recognizing that sentiment is not universal but tied to specific aspects (e.g., "cheap" is good for food but bad for decor) is critical. Building separate sentiment graphs for each aspect allows the model to capture this nuance.
- Unsupervised Seed Set Generation: The ability to automatically derive a relevant seed set of sentiment words using morphological cues and negation is a significant step towards reducing reliance on manual resources. This also suggests better transferability to other languages.
- Handling Nuance in Reviews: The paper highlights how online reviews often contain subtle sentiment expressions, misspellings, and specialized terms that traditional lexicon-based methods would miss. Their unsupervised approach is inherently more robust to these variations.
Key Results
- Aspect Extraction: The local LDA model successfully inferred aspects that aligned with or even distinguished finer granularities than the manually defined aspects in the restaurant domain. For netbooks, it identified relevant aspects like "software" and "memory."
- Sentiment Evaluation: When evaluated against the manually annotated dataset, their system achieved comparable results to semi-supervised methods that used manual seed sets. The paper presents precision-recall curves for the top three aspects (Food, Service, Atmosphere) and reports Kendall's tau coefficients and distance scores, showing good agreement with the gold standard. For instance, the average Kendall's tau coefficient for their automatically derived seed set was 0.36, compared to 0.45 for the manual seed set.
Limitations and Future Work
- Limitations:
- The paper acknowledges that while their automatic seed set performs well, the manually compiled seed set still yields slightly better correlation with the gold standard.
- They note that some aspects, like "cuisine type," are objective and might be considered neutral by annotators but carry sentiment in context.
- The evaluation of sentiment was primarily focused on adjective-noun pairs, which represent a common but not exhaustive way of expressing sentiment.
- Future Work:
- Extend sentiment detection to other parts of speech beyond adjectives.
- Investigate sentiment interactions at a multi-sentence level.
- Develop a full application to test the system's usability in a real-world scenario.
- Explore transferability to other languages more deeply, building on the success of using negation indicators.
Summary updated