SSN_MLRG1 at SemEval-2022 Task 10: Structured Sentiment Analysis using 2-layer BiLSTM

Karun Anantharaman, Divyasri K, Jayannthan Pt, Angel S, Rajalakshmi Sivanaiah, Sakaya Milton Rajendram, Mirnalinee T TNAACL 2022

TL;DR

This paper presents a novel two-layer BiLSTM approach for Structured Sentiment Analysis (SSA), a complex task involving identifying opinion holders, targets, and expressions, along with their relationships and polarity. Their method treats SSA as a sequence labeling problem, achieving an SF1 score of 0.33, and offers a more efficient alternative to traditional graph-based methods.

What Problem Does This Paper Address?

Sentiment analysis, a cornerstone of Natural Language Processing (NLP), aims to extract opinions and emotions from text. However, traditional sentiment analysis often focuses on overall polarity. Structured Sentiment Analysis (SSA) goes a step further by dissecting sentiment into its core components: who is expressing the opinion (holder), what is the opinion about (target), and the actual opinion phrase (expression). Furthermore, it seeks to understand the relationships between these components and their polarity. This paper tackles the challenge of performing SSA efficiently and accurately, moving beyond simple polarity detection to a more granular understanding of sentiment. The authors frame this complex task as a sequence labeling problem, offering a potentially more efficient alternative to existing transition-based and graph-based techniques.

Methods

The core of this work is a two-layer BiLSTM (Bidirectional Long Short-Term Memory) architecture. BiLSTMs are chosen for their ability to capture long-range dependencies in text by processing sequences in both forward and backward directions. The approach treats SSA as a sequence labeling task, where each element in the input text is assigned a label.

Specifically, the model is designed to:

  • Extract Components: Three separate BiLSTM models are trained to identify and extract holders, targets, and expressions.
  • Predict Relationships: A relation prediction model, also utilizing BiLSTM with max pooling, is employed to determine if a relationship exists between the extracted components. This model creates contextualized representations of the full text, the first element (holder or target), and the sentiment expression.
  • Combine and Classify: These representations are concatenated and fed into a linear layer followed by a sigmoid function to predict the relationship. The problem is framed as a binary classification task (relationship exists or not).
  • Sequence Labeling: The entire process leverages the BIO (Beginning, Inside, Outside) sequence labeling scheme to demarcate the different parts of speech within the text.
  • Embeddings: The model utilizes pre-trained GloVe or FastText embeddings to represent words.

Datasets

The study was conducted on seven datasets spanning five languages: English, Spanish, Basque, Norwegian, and Catalan. These datasets were chosen to represent the task's requirements and include:

  • MPQA: A large dataset of news agency text, notable for its high number of holders.
  • NoReCFine: A Norwegian dataset of professional reviews, characterized by a large number of targets and expressions, and also includes intensity of polarity.
  • DSUnis: A dataset comprising user reviews from e-commerce and educational institutions, with a focus on contextual variance in polarity.
  • opeNER project datasets (opener_en, opener_es): Datasets containing targets and expressions.
  • MultiBEU and MultiBCA: Datasets of hotel critiques in Basque and Catalan, respectively, which further qualify polarity.

The paper notes that MPQA and DSUnis are distinctive in including "neutrality" labels, which are less common in other datasets.

Key Insights and Ideas

  • SSA as Sequence Labeling: Framing Structured Sentiment Analysis as a sequence labeling problem is a key methodological innovation, offering a potentially more efficient and scalable approach compared to traditional graph-based methods.
  • Two-Layer BiLSTM Efficacy: The use of a two-layer BiLSTM architecture effectively captures the complex dependencies required for identifying and relating sentiment components.
  • Component-wise Extraction: Separately training BiLSTMs for holders, targets, and expressions allows for specialized learning of each component's characteristics.
  • Efficiency Gain: The proposed method is highlighted as being significantly faster than graph parsing techniques, a crucial factor for real-world applications.

Key Results

The paper reports an SF1 score of 0.33 for their best-performing configuration. This score reflects the model's ability to accurately identify and relate sentiment components. While this score might seem modest, it's important to consider the complexity of the SSA task and the multi-lingual, multi-dataset nature of the evaluation. The authors emphasize that their approach is significantly faster than graph parsing methods.

Limitations and Future Work

  • Polarity Handling: A noted limitation is that the model, in its current form, primarily considers the polarity attributed to the terminal element in complex sentences with multiple expressions. This means it might not fully capture the nuanced sentiment across all expressions within such sentences.
  • Future Directions: The authors suggest exploring the replacement of the BiLSTM component with BiLSTM-CRF (Conditional Random Field). This is motivated by CRF's robustness to word embeddings and its proven capability in tasks like Part-of-Speech tagging, Named Entity Recognition, and chunking, which could potentially lead to improved accuracy.

Summary updated

BibTeX
@inproceedings{anantharaman2022ssn,
  author = {Karun Anantharaman and Divyasri K and Jayannthan Pt and Angel S and Rajalakshmi Sivanaiah and Sakaya Milton Rajendram and Mirnalinee T T},
  title = {{SSN\_MLRG1 at SemEval-2022 Task 10: Structured Sentiment Analysis using 2-layer BiLSTM}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
}
Figure