Learning Explicit and Implicit Structures for Targeted Sentiment Analysis

Hao Li, Wei LuEMNLP 2019 · DOI

TL;DR

This paper introduces a novel model, EI, that jointly predicts target entities and their sentiment in text by effectively integrating both explicit output structures and implicit input structures. EI significantly outperforms existing methods by capturing richer contextual information and more flexible output representations, leading to improved targeted sentiment analysis performance.

What Problem Does This Paper Address?

Targeted Sentiment Analysis (TSA) is the task of identifying specific entities (targets) within a text and determining the sentiment expressed towards them. While existing approaches often treat this as a sequence labeling problem, focusing on explicit output structures, they tend to overlook the crucial implicit structural information present in the input text. This implicit information, which captures the relationships between targets and other words in the sentence, is vital for understanding sentiment. The paper addresses the challenge of jointly modeling both explicit output structures and implicit input structures in a unified framework, especially for the more complex task of jointly predicting targets and their sentiments, where the number of potential target-sentiment combinations can be exponentially large.

Methods

The proposed model, EI (Explicit and Implicit), tackles TSA by combining two key components:

  • Explicit Structure Modeling: Inspired by latent Conditional Random Fields (CRF), EI models flexible sentiment spans in the output. It uses a BMES (Beginning, Middle, End, Single) tagging scheme with sentiment polarity tags (positive, neutral, negative) to represent sentiment spans. This allows for more nuanced and flexible output structures compared to fixed tag sets.
  • Implicit Structure Modeling: To efficiently capture rich implicit structures from the input, EI employs a combination of Bi-directional Long Short-Term Memory (Bi-LSTM) and self-attention mechanisms. The Bi-LSTM encodes contextual features, while self-attention captures the relationships between each word and all other words in the sentence. This allows the model to learn complex dependencies and contextual representations for exponentially many target-sentiment combinations without explicit feature engineering.

The model integrates these two components by assigning scores to edges in the output structure based on both explicit (BMES tags) and implicit (LSTM and self-attention outputs) information.

Datasets

The primary datasets used for training and evaluation are from the work of Mitchell et al. (2013), which include:

  • 2,350 English tweets
  • 7,105 Spanish tweets

Additionally, the paper reports experiments on the multi-lingual Restaurant dataset from SemEval 2016 Task 5, including data for:

  • English
  • Dutch
  • Russian

Key Insights and Ideas

  • The Synergy of Explicit and Implicit Structures: The paper strongly argues and demonstrates that both explicit output structures (how sentiment spans are represented) and implicit input structures (how context influences sentiment) are indispensable for high-performing TSA.
  • Flexibility in Output is Key: Moving beyond fixed output structures (like simple sequence labeling) to more flexible representations (like latent sentiment spans) significantly boosts performance.
  • Self-Attention for Rich Context: Self-attention is shown to be highly effective in capturing nuanced implicit structures, enabling the model to learn complex relationships between targets and sentiment-bearing words, even those far apart in the sentence.
  • Efficiency for Exponential Combinations: The EI model's architecture, particularly the use of LSTM and self-attention, allows it to efficiently handle the exponentially many possible target-sentiment combinations in the joint prediction task.

Key Results

The EI model consistently outperforms all baselines across various metrics (Precision, Recall, F1-score) on both English and Spanish datasets.

  • Overall Performance: EI achieves state-of-the-art results, significantly outperforming strong baselines like EI- (a variant without self-attention) and SA-CRF (a CRF model with self-attention but fixed explicit structures). For instance, on English data, EI obtained a higher F1 score by 6.50 for target recognition and 2.50 for targeted sentiment compared to the best baseline.
  • Robustness: EI demonstrates strong performance across different target lengths, outperforming all baselines on all groups.
  • Subjectivity and Non-Neutral Polarity: On the Spanish dataset, EI achieved higher F1 scores for subjectivity and non-neutral polarities compared to previous best-performing systems.

Limitations and Future Work

  • Limitations:
    • The paper notes that many predicted targets do not match exactly with correct targets, indicating room for improvement in boundary prediction.
  • Errors in sentiment polarity prediction are often due to challenging expressions in tweet text, particularly neutral predictions for positive targets, which are sparse and difficult to learn.
  • Future Work:
    • The authors suggest exploring approaches to capture explicit and implicit structural information for other sentiment analysis tasks and structured prediction problems.
  • Investigating how to better handle challenging linguistic expressions in noisy text like tweets could further improve performance.

Summary updated

BibTeX
@inproceedings{li2019learning,
  author = {Hao Li and Wei Lu},
  title = {{Learning Explicit and Implicit Structures for Targeted Sentiment Analysis}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2019},
  doi = {10.18653/v1/d19-1550},
}
Figure