Sentence-level sentiment analysis based on supervised gradual machine learning

Jing Su, Qun Chen, Yanyan Wang, Lijun Zhang, Wei Pan, Zhanhuai LiScientific 2023 · DOI

Abstract

Sentence-level sentiment analysis (SLSA) aims to identify the overall sentiment polarity conveyed in a given sentence. While deep learning models currently define the state-of-the-art, their reliance on the independent and identically distributed (i.i.d.) assumption often leads to performance degradation in real-world scenarios where training and target data distributions differ. This paper proposes a supervised solution based on the non-i.i.d. paradigm of gradual machine learning (GML). The approach begins with labeled observations and gradually labels target instances in order of increasing difficulty through iterative knowledge conveyance. By leveraging deep neural networks (DNNs) for feature extraction and constructing a factor graph to model relations, the proposed GML framework outperforms pure DNN solutions across multiple benchmarks.

Introduction

Most modern sentiment analysis models rely on the i.i.d. assumption—the premise that training data and real-world target data share the same distribution. However, language is inherently messy and context-dependent. A model trained on movie reviews may struggle with the specific nuances of product feedback or social media posts. Even sophisticated models like BERT can fail on complex sentence structures where local cues (like the word "not") mislead the global sentiment prediction. The authors argue that pure DNN solutions are limited by this distribution misalignment and a lack of explicit "knowledge conveyance" between labeled and unlabeled data.

The SLSA-GML Framework

To address these limitations, the authors introduce a supervised GML solution. Unlike standard classifiers that perform a single-pass prediction, GML operates iteratively, moving from "easy" to "hard" instances.

  1. Feature Extraction via DNNs: The framework uses two specialized DNN architectures to identify relationships between sentences. First, an Affective-aware Polarity Classifier (based on the Entailment as Few-Shot Learner model) creates a polarity-sensitive embedding space. This allows for the identification of K-Nearest Neighbor (KNN) relations. Second, a Semantic Deep Network (built on RoBERTa-Large) is trained to detect implicit sentiment relations (similar or opposite) between arbitrary sentence pairs.
  2. Factor Graph Construction: These relations are modeled as binary factors in a factor graph. KNN factors capture local similarity, while semantic factors capture global or implicit logic.
The SLSA-GML framework: (a) shows the factor graph construction with KNN and semantic factors, and (b) illustrates the iterative gradual inference process.
  1. Gradual Inference: The system identifies unlabeled instances with the highest "evidential certainty" (lowest entropy), labels them, and uses these new labels as evidence for subsequent iterations. This "divide and conquer" strategy allows the model to adapt to the target data distribution on the fly.
Algorithm 1: The Scalable Gradual Inference process used to iteratively label variables based on evidential support and entropy.

Experimental Results

The researchers evaluated their approach on four major sentiment benchmarks: MR (movie reviews), CR (customer reviews), SST (Stanford Sentiment Treebank), and Twitter2013. The GML approach consistently achieved state-of-the-art (SOTA) performance, outperforming models like EFL, SentiLARE, and RoBERTa-Large.

Key accuracy gains over the previous best performer (EFL) included:

  • CR: +1.68%
  • MR: +0.89%
  • SST: +1.76%
  • Twitter2013: +1.58%

Ablation studies confirmed that the synergy between KNN relations and semantic relations is vital. While KNN captures straightforward similarities, the semantic network is crucial for identifying opposite polarities in complex reasoning tasks.

Conclusion

This work demonstrates that while DNNs are excellent feature extractors, their performance can be significantly enhanced when integrated into a non-i.i.d. GML framework. By acting as a bridge between labeled training data and unlabeled target data, GML effectively mitigates distribution misalignment. Future work will explore extending this framework to document-level analysis and multi-label classification, as well as testing its efficacy in weakly supervised settings with minimal labeled data.

Summary updated

Abstract

Sentence-level sentiment analysis (SLSA) aims to identify the overall sentiment polarity conveyed in a given sentence. The state-of-the-art performance of SLSA has been achieved by deep learning models. However, depending on the i.i.d (independent and identically distributed) assumption, the performance of these deep learning models may fall short in real scenarios, where the distributions of training and target data are almost certainly different to some extent. In this paper, we propose a supervised solution based on the non-i.i.d paradigm of gradual machine learning (GML) for SLSA. It begins with some labeled observations, and gradually labels target instances in the order of increasing hardness by iterative knowledge conveyance. It leverages labeled samples for supervised deep feature extraction, and constructs a factor graph based on the extracted features to enable gradual knowledge conveyance. Specifically, it employs a polarity classifier to detect polarity similarity between close neighbors in an embedding space, and a separate binary semantic network to extract implicit polarity relations between arbitrary instances. Our extensive experiments on benchmark datasets show that the proposed approach achieves the state-of-the-art performance on all benchmark datasets. Our work clearly demonstrates that by leveraging DNN for feature extraction, GML can easily outperform the pure DNN solutions.
BibTeX
@inproceedings{su2023sentence,
  author = {Jing Su and Qun Chen and Yanyan Wang and Lijun Zhang and Wei Pan and Zhanhuai Li},
  title = {{Sentence-level sentiment analysis based on supervised gradual machine learning}},
  booktitle = {Scientific},
  year = {2023},
  doi = {10.1038/s41598-023-41485-8},
}
Figure