Auto-ABSA: Automatic Detection of Aspects in Aspect-Based Sentiment Analysis

Teng WangarXiv 2022 · OpenReview

TL;DR

This paper introduces Auto-ABSA, a novel framework for Aspect-Based Sentiment Analysis (ABSA) that automatically detects aspects within a sentence and uses this information to improve sentiment prediction, especially in out-of-domain scenarios. By combining a dedicated aspect detection model (SpanEmo) with a sentiment predictor, Auto-ABSA significantly boosts performance without requiring manual aspect annotations.

What Problem Does This Paper Address?

Sentiment analysis, a key NLP task, aims to understand the emotional tone of text. While basic sentiment analysis focuses on the overall sentiment of a sentence, Aspect-Based Sentiment Analysis (ABSA) goes deeper by identifying specific aspects (e.g., "food," "service") within a sentence and determining the sentiment expressed towards each aspect. This is crucial because a single sentence can contain multiple aspects with differing sentiments (e.g., "The food was great, but the service was slow").

A major challenge in ABSA is the need for accurate aspect identification. Traditional methods often require manual annotation of aspects, which is time-consuming and expensive, especially when dealing with new domains where existing labeled data is scarce. Furthermore, adapting ABSA models to new domains (out-of-domain ABSA) is difficult because aspect terminology and sentiment expressions can vary significantly across domains. This paper tackles the problem of performing effective ABSA without relying on domain-specific labeled data or manual aspect annotations.

Methods

The proposed Auto-ABSA framework consists of two main components:

  1. SpanEmo for Aspect Detection: The authors adapt SpanEmo, a model originally designed for multi-label emotion classification, to perform multi-aspect detection. SpanEmo takes a sentence and a set of predefined aspect classes as input and outputs the likelihood of each aspect being present in the sentence. This allows for the automatic identification of relevant aspects.
  2. Sentiment Predictor: A sentiment classification model is developed to predict sentiment polarity (positive, negative, neutral). This predictor is designed to generalize across domains. The key innovation is how it's used:
  3. SpanEmo + Sentiment Predictor: The most comprehensive approach where SpanEmo first identifies aspects, which are then fed along with the original sentence to the sentiment predictor.
  4. Sentiment Predictor with All Aspects: A baseline where all possible aspects are provided to the sentiment predictor, bypassing the automatic detection step.
  5. Sentiment Predictor without Aspects: Another baseline where no aspect information is provided, relying solely on the sentence for sentiment prediction.

The framework is designed for out-of-domain ABSA by training SpanEmo on one dataset and then using the integrated "Big Model" (SpanEmo + Sentiment Predictor) to perform ABSA on a different, unseen domain.

Datasets

The paper utilizes four publicly available datasets for training, evaluation, and testing:

  • SentiHood: An urban neighborhood reviews dataset with aspect categories and sentiment polarities.
  • SemEval-2014 Task 4: A benchmark for ABSA in restaurant reviews.
  • SemEval-2015 Task 12: Focuses on more fine-grained aspect categories.
  • SemEval-2016 Task 5: Targets ABSA in restaurant and laptop domains.

These datasets cover diverse domains and aspect categories, allowing for a thorough evaluation of the model's generalizability.

Key Insights and Ideas

  • Automatic Aspect Detection is Crucial: The experiments strongly suggest that accurately identifying aspects is paramount for effective ABSA, especially in out-of-domain scenarios. Providing correct aspect information significantly boosts sentiment prediction performance.
  • Auxiliary Sentences Enhance Understanding: The use of auxiliary sentences (generated by SpanEmo) to represent detected aspects helps the sentiment predictor focus on relevant parts of the text, leading to improved accuracy.
  • Out-of-Domain Generalization is Achievable: The "Big Model" demonstrates a promising ability to perform ABSA on unseen domains without requiring domain-specific training data, highlighting the power of pre-trained models and effective aspect integration.
  • "Big Model" Can Outperform Ground Truth: Interestingly, the integrated "Big Model" sometimes achieved better results than providing the sentiment predictor with actual, correct aspects. This suggests the model might be better at mitigating domain-specific biases present in the training data by relying on its own predicted aspects.

Key Results

  • Aspect Detection Performance: SpanEmo achieved competitive aspect detection performance across various datasets, with RoBERTa-base and DeBERTa-base showing strong results on different benchmarks.
  • Sentiment Prediction Improvement: Providing correct aspects to the sentiment predictor significantly improved F1 scores compared to using all aspects or no aspects at all. For instance, in one experiment, the "With Correct Aspects" setting yielded higher F1 scores than "Without Aspects."
  • Out-of-Domain ABSA Effectiveness: The "Big Model" (SpanEmo + Sentiment Predictor) demonstrated strong performance in out-of-domain ABSA tasks, often outperforming baselines that relied on either correct or all aspects. For example, on the SemEval-2016 dataset, the Big Model achieved F1-Micro scores around 0.74-0.76, depending on the underlying pre-trained model.

Limitations and Future Work

  • Reliance on Aspect Detection: The performance of the sentiment predictor is directly tied to the accuracy of the aspect detection model. Errors in aspect detection can propagate and negatively impact sentiment prediction.
  • Pre-trained Model Biases: While the method reduces the need for domain-specific data, the underlying pre-trained language models might still carry domain biases learned during their initial training.

Future work suggested includes:

  • Domain Adaptation Techniques: Exploring adversarial training or fine-tuning with small amounts of domain-specific data to improve robustness.
  • Multi-Task Learning: Jointly training aspect detection and sentiment prediction tasks to leverage shared representations.
  • External Knowledge Integration: Incorporating knowledge bases or ontologies to enrich the model's understanding of aspects and sentiments.
  • Error Analysis: Conducting detailed error analysis to identify common failure modes and refine the model.

Summary updated

Abstract

After transformer is proposed, lots of pre-trained language models have been come up with and sentiment analysis (SA) task has been improved. In this paper, we proposed a method that uses an auxiliary sentence about aspects that the sentence contains to help sentiment prediction. The first is aspect detection, which uses a multi-aspects detection model to predict all aspects that the sentence has. Combining the predicted aspects and the original sentence as Sentiment Analysis (SA) model's input. The second is to do out-of-domain aspect-based sentiment analysis(ABSA), train sentiment classification model with one kind of dataset and validate it with another kind of dataset. Finally, we created two baselines, they use no aspect and all aspects as sentiment classification model's input, respectively. Compare two baselines performance to our method, found that our method really makes sense.
BibTeX
@misc{wang2022auto,
  author = {Teng Wang},
  title = {{Auto-ABSA: Automatic Detection of Aspects in Aspect-Based Sentiment Analysis}},
  year = {2022},
  eprint = {2202.00484},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2202.00484},
}
Figure