DNN-driven Gradual Machine Learning for Aspect-term Sentiment Analysis

Murtadha Ahmed, Qun Chen, Yanyan Wang, Youcef Nafa, Zhanhuai Li, Tianyi DuanACL 2021

TL;DR

This paper introduces a novel DNN-driven Gradual Machine Learning (GML) approach for Aspect-Term Sentiment Analysis (ATSA). By leveraging deep neural networks for feature representation and an unsupervised clustering mechanism, the method effectively captures implicit sentiment features, leading to state-of-the-art performance in both supervised and unsupervised settings.

What Problem Does This Paper Address?

Aspect-Term Sentiment Analysis (ATSA) aims to determine the sentiment (positive, negative, neutral) expressed towards specific aspects within a sentence. For example, in "The food was delicious, but the service was slow," ATSA would identify the sentiment towards "food" as positive and towards "service" as negative. While traditional machine learning and deep learning models have made strides, they often require large amounts of labeled data, which is expensive and time-consuming to obtain. Gradual Machine Learning (GML) offers a promising alternative by starting with easily labeled instances and iteratively inferring labels for more challenging ones. However, existing GML approaches for ATSA rely on pre-defined sentiment lexicons, which are often incomplete and struggle to capture nuanced or implicit sentiment cues. This paper addresses the limitation of existing GML methods by proposing a DNN-driven approach that can automatically learn and leverage these implicit sentiment features.

Methods

The core of this work is a DNN-driven GML framework for ATSA. The approach consists of several key components:

  • Implicit Feature Extraction by DNN: The authors propose an unsupervised neural network that leverages deep learning to extract sentiment features. This network learns "polarity indicators" that represent clusters of features commonly associated with the same sentiment orientation. It uses an attention mechanism to down-weight irrelevant words and focus on sentiment-bearing words related to the aspect. The network is trained to reconstruct sentence representations from these learned polarity indicators.
  • Unsupervised GML Solution: Once polarity indicators are learned, they are used as implicit features for the GML process. For each sentence and aspect, a vector representation is computed. The similarity between this vector and each polarity indicator is calculated, generating scores that represent how well the sentence aligns with each sentiment cluster. These scores are then used as features in a factor graph for gradual inference.
  • Hybrid GML Solution: To further enhance performance, a hybrid approach is introduced. This method combines the implicit features learned by the DNN with explicit predictions from a supervised DNN model (specifically, HP-SUM, a BERT-based model). This fusion allows the model to leverage both the power of supervised learning and the implicit knowledge conveyance of GML, using supervised predictions to identify "easy" instances to kick-start the gradual learning process.

Datasets

The paper evaluates its approach on six benchmark datasets from the SemEval ABSA task across the years 2014, 2015, and 2016, specifically for the Restaurant and Laptop domains. The experiments focus on binary polarity classification (positive/negative), excluding neutral labels. For unsupervised training of polarity indicators, the authors mention using unlabeled corpora collected from CitySearch and Amazon, which have been used in prior work.

Key Insights and Ideas

  • Implicit Features are Crucial: The research highlights that relying solely on explicit sentiment lexicons is insufficient for ATSA. Implicit sentiment, conveyed through word associations and contextual nuances, plays a significant role and can be effectively captured by DNNs.
  • DNNs as Feature Extractors for GML: Deep neural networks are not just used for end-to-end sentiment analysis but are repurposed as powerful feature extractors for the GML paradigm, enabling knowledge conveyance beyond explicit lexicon matches.
  • Unsupervised Clustering for Sentiment: The unsupervised neural network's ability to cluster features based on sentiment orientation is a key innovation, allowing the model to discover sentiment patterns without manual labeling.
  • Hybrid Approach Synergizes Strengths: Combining supervised DNN predictions with unsupervised GML offers a robust solution, leveraging the accuracy of supervised methods for initial labeling and the efficiency of GML for handling larger, unlabeled datasets.

Key Results

The empirical evaluation demonstrates significant improvements:

  • Unsupervised Performance: The proposed unsupervised DNN-driven GML consistently outperforms other unsupervised methods, including lexicon-based GML, across all datasets.
  • Competitive with Supervised Models: The unsupervised DNN-driven GML achieves performance comparable to traditional supervised attention-based models.
  • State-of-the-Art with Hybrid Approach: The hybrid GML solution consistently achieves state-of-the-art performance across all datasets, improving upon the best existing methods by approximately 1-2%.

Limitations and Future Work

  • Limitations: The paper acknowledges that the efficacy of the hybrid approach relies on the availability of labeled data for the initial supervised predictions. While GML reduces the overall labeling burden, the initial supervised phase still requires some labeled data.
  • Future Work: The authors suggest that the DNN-driven GML approach is potentially applicable to other classification tasks beyond ATSA, and further investigation into its technical solutions for broader applications is warranted.

Summary updated

BibTeX
@inproceedings{ahmed2021dnn,
  author = {Murtadha Ahmed and Qun Chen and Yanyan Wang and Youcef Nafa and Zhanhuai Li and Tianyi Duan},
  title = {{DNN-driven Gradual Machine Learning for Aspect-term Sentiment Analysis}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
}
Figure