TL;DR
This paper explores graph-based algorithms for sentiment classification across different domains (e.g., book reviews vs. electronics reviews). They find that a ranking-based graph algorithm (RANK) generally outperforms an optimization-based one (OPTIM) and other state-of-the-art methods, especially when using domain-specific sentiment dictionaries. Crucially, they demonstrate that optimal algorithm parameters are domain-dependent, not universal.
What Problem Does This Paper Address?
Sentiment classification (SC) aims to automatically determine the emotional tone of text. While effective for specific product categories (domains), models trained on one domain often perform poorly on another due to domain-specific language and sentiment expression. For instance, "ridiculous" might be negative for a book but positive for an electronics price. This "domain adaptation" problem is challenging. When labeled data is scarce in a target domain, researchers turn to cross-domain approaches to leverage abundant labeled data from other domains. This paper specifically investigates how well graph-based methods, which model documents and their relationships, can tackle this cross-domain sentiment classification challenge.
Methods
The paper compares two graph-based algorithms:
- OPTIM (Goldberg & Zhu, 2006): This method frames sentiment inference as an optimization problem. It aims to find a sentiment function that is smooth across connected documents in a graph, minimizing the difference between predicted and actual ratings for labeled data and maintaining consistency between neighbors in unlabeled data.
- RANK (Wu et al., 2009): This algorithm uses a similar graph structure but employs an iterative ranking approach. It updates sentiment scores for unlabeled documents based on a weighted sum of scores from their labeled and unlabeled neighbors. The key difference from OPTIM is that it doesn't require initial sentiment scores to be close to final scores.
The authors also explore different measures of document similarity to build the graph edges, including:
- Feature-based: Using weighted document features (like TF-IDF) based on unigrams and bigrams, and an extension that incorporates sentiment dictionaries for nouns.
- Lexicon-based: Using sentiment dictionaries (like SO-CAL) to assign scores to words or sentences.
Datasets
The experiments were conducted on Amazon product reviews from four domains:
- Books (BO)
- Electronics (EL)
- Kitchen Appliances (KI)
- DVDs (DV)
Each domain contained 1000 positive (4-5 star) and 1000 negative (1-2 star) reviews, balanced for binary sentiment classification.
Key Insights and Ideas
- Graph-based methods are competitive: The paper confirms that graph-based representations, which capture document relationships beyond just content, offer a viable solution for cross-domain sentiment classification.
- RANK outperforms OPTIM: The iterative ranking approach (RANK) generally yields better results than the optimization-based approach (OPTIM), especially for dissimilar domains.
- Domain-specific sentiment dictionaries are crucial: Integrating sentiment dictionaries (like SO-CAL) significantly boosts the performance of graph-based algorithms, highlighting the importance of domain-specific vocabulary.
- No universal parameters: The most surprising finding is that there are no "one-size-fits-all" optimal parameter values (like the number of neighbors or weighting factors) for these graph algorithms. The best parameters are highly dependent on the specific characteristics of the domain pair being analyzed.
- Domain similarity and complexity matter: The parameter
γin the RANK algorithm shows a strong correlation with domain similarity and complexity, suggesting these characteristics can guide parameter selection.
Key Results
- RANK vs. OPTIM: RANK consistently outperformed OPTIM across most domain pairs, with OPTIM showing competitive performance only for similar domains.
- RANK vs. SCL/SFA: RANK significantly outperformed Structural Correspondence Learning (SCL) and generally achieved better results than Spectral Feature Alignment (SFA), especially when using SO-CAL dictionaries. Without these dictionaries, RANK's performance dropped.
- Impact of SO-CAL: Integrating SO-CAL dictionaries improved accuracies by an average of ~3% for RANK and ~1.5% for OPTIM.
- Parameter Dependency: Table 3 shows varying optimal parameter values for
k(labeled neighbors),k'(unlabeled neighbors), andγacross different domain pairs, reinforcing the lack of universal settings.
Limitations and Future Work
- Parameter Optimization Strategy: While the paper demonstrates the domain-dependency of parameters, a fully automated strategy for choosing optimal parameters based on domain characteristics is still needed. The authors suggest their previous work on domain similarity and complexity can be a starting point.
- "Bad" Neighbors: The current approach of selecting neighbors based on quantitative thresholds might include irrelevant "bad" neighbors. Future work aims to explore qualitative thresholds and similarity measures that are consistent across in-domain and cross-domain graphs.
- Lexical Discrepancy: Integrating Spectral Feature Alignment (SFA) into the similarity measure is proposed to better handle lexical differences between domains.
- Dataset Size: Conclusions are drawn from a dataset of 12 domain pairs. Expanding to a larger number of domains is planned to verify findings.
Summary updated