Sentiment Expression Boundaries in Sentiment Polarity Classification

Rasoul Kaljahi, Jennifer FosterEMNLP 2018

TL;DR

This paper introduces the concept of "sentiment expression boundaries" – the specific words in a sentence that convey sentiment towards an aspect. By annotating these boundaries and incorporating them into neural network models, the authors demonstrate a significant improvement in aspect-based sentiment polarity classification accuracy. Their findings suggest that focusing on these precise sentiment-carrying phrases is more effective than using the entire sentence or relying solely on general word embeddings.

What Problem Does This Paper Address?

Aspect-based sentiment analysis (ABSA) aims to determine the sentiment (positive, negative, neutral) expressed towards specific aspects within a text, going beyond a single overall sentiment score. While effective, existing methods often treat the entire sentence as input, potentially diluting the impact of the actual sentiment-carrying words. This paper addresses the challenge of pinpointing which words within a sentence are responsible for expressing sentiment towards a given aspect, hypothesizing that this fine-grained information can significantly enhance ABSA performance. The authors aim to improve sentence-level aspect-based sentiment polarity classification by explicitly identifying and leveraging these "sentiment expression boundaries."

Methods

The core of this work lies in two main contributions:

  1. Manual Annotation of Sentiment Expression Boundaries: The authors manually annotated a subset of the SemEval 2016 Task 5 dataset to identify the specific words (sentiment expressions) that convey sentiment towards an aspect. This annotation process involved two annotators and a rigorous guideline calibration to ensure consistency.
  2. Integration of Sentiment Expressions into Neural Models: They experimented with several neural network architectures, primarily focusing on Long Short-Term Memory (LSTM) and Convolutional Neural Networks (CNNs), and their combinations. The sentiment expression information was integrated in two primary ways:
  3. Auxiliary Input: Sentiment expression boundaries were encoded as binary flags (1 for sentiment expression, 0 otherwise) and fed into the model alongside word embeddings and aspect category information.
  4. Filtering: Only the sentiment expression tokens were used as input to the model, effectively filtering out non-sentiment-carrying words.

They also explored a multitask learning approach where sentiment expression extraction was trained jointly with polarity classification.

Datasets

The primary dataset used in this research is the SemEval 2016 Task on Aspect-Based Sentiment Analysis (Task 5, Subtask 1). This dataset consists of consumer reviews of restaurants and laptops, with sentences annotated for aspect categories and sentiment polarity. The authors also created a subset of this dataset with manual annotations for sentiment expression boundaries.

Key Insights and Ideas

  • The Power of Precision: Explicitly identifying the words that carry sentiment ("sentiment expression boundaries") provides a significant boost to polarity classification accuracy. This highlights that not all words in a sentence are equally important for determining sentiment towards a specific aspect.
  • CNNs Outperform LSTMs (on this task): For the baseline models, CNN architectures generally achieved better performance than LSTMs on this specific dataset and task.
  • Auxiliary Input is Key: Using sentiment expression boundaries as an auxiliary input to the model proved more effective than simply filtering out non-sentiment tokens. This suggests that the context of non-sentiment words can still be indirectly useful, or that the binary flag representation is a more robust way to inject this information.
  • Multitask Learning Limitations: While multitask learning is a promising avenue for related tasks, their implementation for jointly learning sentiment expression boundaries and polarity did not yield improvements over the baseline models. This might indicate that the tasks are not as tightly coupled as initially hypothesized or that the specific architecture needs further refinement.
  • Improved Handling of Neutral Cases: Models augmented with sentiment expression information showed better performance on neutral sentiment cases, suggesting they can better associate the absence of explicit sentiment expressions with neutrality.

Key Results

  • Significant Accuracy Boost: Incorporating sentiment expression boundaries as auxiliary input led to an average improvement of six percentage points in polarity classification accuracy across datasets.
  • CNN vs. LSTM: CNN models generally outperformed LSTM models in baseline experiments.
  • Combined Architectures: While combining CNNs and LSTMs showed promise, the specific combinations explored did not consistently outperform the best individual CNN models.
  • Inter-Annotator Agreement: High inter-annotator agreement (precision, recall, and F1 scores) was achieved for the sentiment expression annotations, indicating a reasonable level of consistency in the manual labeling.

Limitations and Future Work

  • Manual Annotation Cost: The primary limitation is the manual effort required for annotating sentiment expression boundaries, which is time-consuming and expensive.
  • Multitask Learning Performance: The multitask learning approach did not yield improvements, suggesting further research is needed to optimize this strategy.
  • Future Work: The authors propose developing an automatic sentiment expression extraction system and then using these automatically extracted boundaries in a pipeline approach for polarity classification. They also plan to investigate further how linguistic insights from these annotations can be leveraged. The availability of their annotated dataset is also highlighted as a resource for future research.

Summary updated

BibTeX
@inproceedings{kaljahi2018sentiment,
  author = {Rasoul Kaljahi and Jennifer Foster},
  title = {{Sentiment Expression Boundaries in Sentiment Polarity Classification}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
}
Figure