LSA: Modeling Aspect Sentiment Coherency via Local Sentiment Aggregation

Heng Yang, Ke Li

Abstract

Aspect sentiment coherency is an intriguing yet underexplored topic in the field of aspect-based sentiment classification. This concept reflects the common pattern where adjacent aspects often share similar sentiments. Despite its prevalence, current studies have not fully recognized the potential of modeling aspect sentiment coherency, including its implications in adversarial defense. To model aspect sentiment coherency, we propose a novel local sentiment aggregation (LSA) paradigm based on constructing a differential-weighted sentiment aggregation window. We have rigorously evaluated our model through experiments, and the results affirm the proficiency of LSA in terms of aspect coherency prediction and aspect sentiment classification. For instance, it outperforms existing models and achieves state-of-the-art sentiment classification performance across five public datasets. Furthermore, we demonstrate the promising ability of LSA in ABSC adversarial defense, thanks to its sentiment coherency modeling.

Introduction: The Challenge of Aspect-Based Sentiment

In Aspect-Based Sentiment Classification (ABSC), the objective is to determine the sentiment toward specific entities within a sentence. For example, in the sentence "The screen is brilliant but the battery life is short," a model must identify "screen" as positive and "battery life" as negative.

Traditional approaches often treat these aspects as independent tasks or rely on complex, computationally expensive syntax trees to map linguistic dependencies. However, these methods frequently overlook a simple yet pervasive pattern in natural language: sentiment coherency. Humans often group similar sentiments together, creating "vibe" consistency across adjacent aspects.

The Concept of Aspect Sentiment Coherency

Sentiment coherency refers to the observation that adjacent aspects in a text (like "food" and "service" in a restaurant review) usually share the same sentiment polarity. The authors demonstrate that a vast majority of aspects in public datasets belong to "sentiment clusters" where neighbors share the same sentiment.

Illustration of sentiment clusters and local coherency in a review sentence, where adjacent aspects like "food" and "service" share sentiment.

Existing models struggle to leverage this consistency, often remaining fragile to small adversarial changes. By modeling this coherency, we can create more robust classifiers that "borrow" sentiment signals from neighboring aspects when the local signal is ambiguous.

The Local Sentiment Aggregation (LSA) Framework

To address these limitations, we propose Local Sentiment Aggregation (LSA). Instead of analyzing an aspect in isolation, LSA constructs a "sentiment aggregation window" that includes the target aspect and its $k$ nearest neighbors (typically $k=1$).

The LSA architecture, illustrating how features from a Pre-trained Language Model (PLM) are processed into aspect-focused features and aggregated with global context.

Core Components

  1. Feature Learning: The model utilizes Pre-trained Language Models (PLMs) like BERT or DeBERTa. We introduce three variants: $LSA_P$ (standard sentence-pair), $LSA_T$ (local context weighting), and $LSA_S$ (syntax-aware weighting).
  2. Differential Weighted Aggregation (DWA): Recognizing that not all neighbors are equally informative, LSA uses learnable weights to dynamically determine the influence of left and right adjacent aspects on the target aspect's final representation.
  3. Window Padding: For single-aspect sentences, a "copy" strategy is used to maintain the window structure without introducing noise.

Key Results and Robustness

LSA achieves state-of-the-art (SOTA) results across five primary benchmark datasets (Laptop14, Restaurant14, 15, 16, and MAMS). Notably, the model excels in "Exact Match" (EM) tasks for predicting sentiment clusters, proving it effectively captures coherency.

Beyond accuracy, LSA provides a significant boost to adversarial defense. On the ARTS (Adversarial Robustness Test Set), LSA models showed massive improvements over baselines—for instance, jumping from a baseline BERT accuracy of ~64% to ~74% on Laptop14-ARTS. Modeling coherency acts as a "smoothing" mechanism; even if an attacker perturbs the words around one aspect, the stable sentiment of adjacent aspects helps the model maintain the correct prediction.

Conclusion

LSA offers a simple yet powerful paradigm for ABSC by leveraging the natural tendency of sentiment to cluster locally. By moving away from heavy computational dependencies like deep Graph Convolutional Networks (GCNs) and focusing on local aggregation, LSA provides a more efficient and robust path forward for sentiment analysis.

Summary updated

Abstract

Aspect sentiment coherency is an intriguing yet underexplored topic in the field of aspect-based sentiment classification. This concept reflects the common pattern where adjacent aspects often share similar sentiments. Despite its prevalence, current studies have not fully recognized the potential of modeling aspect sentiment coherency, including its implications in adversarial defense. To model aspect sentiment coherency, we propose a novel local sentiment aggregation (LSA) paradigm based on constructing a differential-weighted sentiment aggregation window. We have rigorously evaluated our model through experiments, and the results affirm the proficiency of LSA in terms of aspect coherency prediction and aspect sentiment classification. For instance, it outperforms existing models and achieves state-of-the-art sentiment classification performance across five public datasets. Furthermore, we demonstrate the promising ability of LSA in ABSC adversarial defense, thanks to its sentiment coherency modeling. To encourage further exploration and application of this concept, we have made our code publicly accessible. This will provide researchers with a valuable tool to delve into sentiment coherency modeling in future research.
BibTeX
@inproceedings{yang2024lsa,
  author = {Heng Yang and Ke Li},
  title = {{LSA: Modeling Aspect Sentiment Coherency via Local Sentiment Aggregation}},
  booktitle = {EACL (Findings)},
  year = {2024},
}
Figure