TL;DR
This paper introduces OPINE, an unsupervised system for extracting product features and customer opinions from online reviews. OPINE achieves state-of-the-art precision in feature extraction by novelly combining web-scale statistics with a relaxation labeling approach to understand opinion sentiment in context.
What Problem Does This Paper Address?
Online product reviews are a treasure trove of information, but wading through them to find specific opinions about product features is a significant challenge for consumers. While existing "opinion mining" systems can identify product features and general sentiment, they often struggle with:
- Identifying specific product features: Distinguishing between general product mentions and actual features customers care about.
- Linking opinions to features: Accurately associating a sentiment (positive/negative) with the correct product feature within a sentence.
- Understanding context-dependent sentiment: Recognizing that the same word can have different sentiment depending on the feature and sentence it's used with (e.g., "hot room" vs. "hot water").
This paper aims to address these limitations by building a system that can automatically extract a structured representation of product features and the opinions expressed about them, ranked by strength.
Methods
OPINE is built upon the KnowItAll information extraction system and tackles the problem in several stages:
- Feature Extraction: OPINE identifies explicit product features by first extracting noun phrases and then using a "Feature Assessor" that leverages Pointwise Mutual Information (PMI) with domain-specific phrases and web statistics to score candidate features.
- Opinion Phrase Identification: Potential opinion phrases are identified based on their proximity to extracted features, using syntactic dependencies from the MINIPAR parser.
- Semantic Orientation (SO) Determination: This is OPINE's core innovation. It uses a three-step relaxation labeling process:
- Word SO: Assigns an initial sentiment (positive, negative, neutral) to individual words based on PMI with positive/negative keywords.
- Word-Feature SO: Refines word sentiment based on its association with specific features, recognizing that words can change sentiment depending on the feature (e.g., "hot" in "hot water" vs. "hot room").
- Word-Feature-Sentence SO: The most granular step, where sentiment is determined within the specific sentence context, crucially incorporating negation modifiers.
Relaxation labeling is a computer vision technique that iteratively refines label assignments (sentiment in this case) by considering the influence of neighboring objects and their labels, guided by various constraint sources (conjunctions, syntactic rules, WordNet relationships).
Datasets
The paper evaluates OPINE on several datasets:
- Hu and Liu (2004) datasets: Publicly available datasets for 5 product classes used for comparison.
- Tripadvisor.com (Hotels) and Amazon.com (Scanners) datasets: Used for manual annotation and evaluation of explicit feature extraction.
- Annotated sentences and features: A set of 13841 sentences with 538 pre-extracted features were used for evaluating opinion phrase extraction and polarity.
Key Insights and Ideas
- Web PMI is a powerful feature assessor: Leveraging web search engine statistics (PMI) significantly boosts the precision of identifying relevant product features, contributing two-thirds of the precision improvement over previous methods.
- Relaxation labeling is key for context-sensitive sentiment: The iterative nature of relaxation labeling allows OPINE to effectively capture how a word's sentiment changes based on the specific product feature and sentence it appears in, a critical limitation of prior work.
- Opinion mining is a multi-stage problem: Breaking down the task into feature extraction, opinion phrase identification, and sentiment determination, with each stage building on the previous, is a robust approach.
- Syntactic dependencies are crucial for opinion phrase identification: Moving beyond simple window-based approaches, using parser output to link opinions to features is more precise.
Key Results
- Feature Extraction: OPINE achieved 22% higher precision than the state-of-the-art Hu and Liu (2004) system on their datasets, with only a 3% drop in recall. Manual annotation on new datasets showed 89% precision for explicit feature extraction.
- Opinion Phrase Extraction: OPINE achieved 79% precision and 76% recall.
- Opinion Phrase Polarity Determination: OPINE achieved 86% precision and 89% recall.
- Semantic Orientation (SO) for Words: OPINE demonstrated higher precision than baseline methods (PMI++, Hu++) across different word types (adjectives, nouns, verbs, adverbs), with a slight loss in recall.
Limitations and Future Work
- Recall in Feature Extraction: While precision is high, there's a slight drop in recall for feature extraction compared to Hu and Liu's system.
- Implicit Features and Opinion Ranking: The paper focuses on explicit features and does not detail the mechanisms for identifying implicit features or ranking opinions by strength, though these are mentioned as part of the overall OPINE system.
- Future Work: The authors suggest extending the system to handle implicit features and opinion ranking, and applying the methods to broader domains beyond product reviews.
Summary updated