Aspect Based Sentiment Analysis using Sentiment Flow with Local and Non-local Neighbor Information

Shubham PateriaCOLING 2016

TL;DR

This paper introduces a novel two-level model for Aspect-Based Sentiment Analysis (ABSA) that leverages both local and non-local contextual information within reviews. By first predicting sentiment probabilities and then using these probabilities to inform a second-level classifier, the model significantly improves accuracy, particularly in handling ambiguous sentiments.

What Problem Does This Paper Address?

Consumers increasingly express their opinions through online reviews, making it crucial to understand their sentiments towards specific product or service aspects. While traditional Aspect-Based Sentiment Analysis (ABSA) methods often focus on local context around an aspect, they struggle with ambiguities and the richer context provided by non-local information. This paper addresses the challenge of effectively incorporating this non-local information to improve sentiment prediction accuracy, especially in reviews with complex discourse and sentiment shifts.

Methods

The core contribution is a two-level model for ABSA.

  • Level-1 (Base Prediction): This stage uses a Support Vector Machine (SVM) classifier with a Radial Basis Function (RBF) kernel. It's trained on a comprehensive set of features including sentiment scores from lexicons (Bing Liu, MPQA, SentiWordNet), category-specific keywords, neutral terms, bi-grams (especially around negation), and aspect-category type. Crucially, this level outputs not just a predicted sentiment label but also probability estimates for each class. These probabilities serve as confidence scores and are used to identify ambiguous sentiments.
  • Level-2 (Non-local Context Integration): This stage builds upon the Level-1 predictions. Neighboring aspect units (both local and non-local) are grouped based on their predicted sentiments and confidence scores. New features are engineered to capture information from these groups, including their polarity, count, and distance. These features, along with the original Level-1 features, are fed into a second SVM classifier. This approach allows the model to learn from the broader context of the review, resolving ambiguities by considering how sentiments flow across different aspects.

The authors also experimented with a linear-chain Conditional Random Field (CRF) at Level-2 to explore structured prediction, though it was not directly compared against the SVM-based Level-2 model due to setup differences.

Datasets

The paper evaluates its model on two main sources:

  • SemEval Workshops (2015 & 2016): Specifically, data from the Aspect Based Sentiment Analysis (ABSA) tasks. This data is provided for the Restaurant domain and includes labels for aspect-terms, categories, and polarity.
  • Bing Liu's Consumer Review Collection: This dataset, comprising 5 + 9 product datasets for the Consumer Electronics (CE) domain, provides ordinal sentiment labels (-3 to +3) which were mapped to three classes (positive, negative, neutral).

The data was split into training and evaluation sets, with specific ratios mentioned for Bing Liu's data (70:30). The SemEval 2016 training data was a mix of SemEval 2015 training and evaluation data.

Key Insights and Ideas

  • Sentiment Flow Matters: The paper highlights that sentiment isn't isolated to local context. The "flow" of sentiment across different aspects in a review provides crucial information for accurate prediction, especially when dealing with ambiguous statements.
  • Probabilities as Bridges: Using the probability estimates from a base classifier (Level-1) is a clever way to quantify uncertainty and provide a "soft" signal for downstream processing. This allows the model to learn from both confident and uncertain predictions.
  • Grouping for Non-local Context: Grouping aspect units based on Level-1 predictions effectively aggregates information from distant parts of the review, making non-local context manageable and useful for the Level-2 classifier.
  • Ambiguity Detection: The proposed ambiguity criteria, based on the difference between probability values, is a practical way to identify sentences or aspects that require more careful consideration, leveraging the output of the base model.

Key Results

The proposed two-stage model ("Base + Level-2") consistently outperformed the "Base Model" alone across both datasets.

  • Accuracy Improvements: The combined model achieved accuracies in the range of 83-88%. Specifically, it showed an improvement of approximately 3-4 points over the baseline (Level-1 only) scores.
  • Dataset Performance: The model performed relatively better on the SemEval-2015 data. For SemEval-2016, the top-scoring system (XRCE) achieved a higher accuracy (88.12%) than the proposed model (87.30%), which the authors attribute to XRCE's feedback mechanism for feature design.
  • CRF vs. SVM: The Level-2 CRF model showed lower accuracy than the Level-2 SVM model, which the authors suggest might be due to the linear-chain nature of the CRF and differences in the prediction setup.

Limitations and Future Work

  • Limitations:
    • The CRF experiment used a linear-chain model, which might not fully capture long-distance dependencies.
  • Direct comparison between the SVM and CRF Level-2 models was difficult due to differing prediction setups.
  • The authors acknowledge that their approach to incorporating non-local context is "rudimentary."
    • The top-scoring system on SemEval-2016 used a more sophisticated feedback mechanism for feature design, which was not explored here.
  • Future Work:
    • Devising more efficient structured prediction models that can handle long-distance dependencies beyond linear chains.
  • Exploring CRF with Posterior Regularization constraints, building on prior work.
  • Aggregating sentiment at sentence or phrase levels using discourse markers.
  • Expanding the use of richer discourse relations and exploring their application in linking aspect units.
  • Investigating more advanced techniques for feature relevance and design, inspired by top-performing systems.

Summary updated

BibTeX
@inproceedings{pateria2016aspect,
  author = {Shubham Pateria},
  title = {{Aspect Based Sentiment Analysis using Sentiment Flow with Local and Non-local Neighbor Information}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2016},
}
Figure