Modeling Aspect Sentiment Coherency via Local Sentiment Aggregation

Heng Yang, Ke Li

Abstract

Aspect-Based Sentiment Classification (ABSC) aims to identify sentiments associated with specific entities in text. Traditional methods either treat aspects in isolation or rely on complex syntactical dependencies. This paper introduces Local Sentiment Aggregation (LSA), a paradigm based on the observation of aspect sentiment coherency—the tendency for neighboring aspects to share the same sentiment. By aggregating local sentiment signals, LSA achieves state-of-the-art performance and enhanced robustness against adversarial attacks.

Introduction

In ABSC, identifying the sentiment of multiple aspects within a single sentence (e.g., "food" vs. "service") is a core challenge. Current research typically follows two paths: isolating aspects to avoid interference or utilizing heavy syntax trees to map word dependencies. However, these approaches often ignore a fundamental linguistic pattern: coherency. Users frequently group similar sentiments together in reviews. Failing to model this "stickiness" results in models that are sensitive to noise and miss valuable contextual signals.

The Concept of Aspect Sentiment Coherency

The authors define aspect sentiment coherency as the local clustering of sentiments. Statistical analysis of datasets like MAMS reveals that nearly half of all aspects belong to sentiment clusters of size two or larger. This suggests that the sentiment of a target aspect is highly correlated with its immediate neighbors.

Illustration of sentiment clusters and local coherency within a sample review sentence.

As shown in the figure above, terms like "atmosphere," "food," and "service" often form a coherent sentiment cluster, providing a collective signal that can be used to reinforce individual aspect predictions.

Methods: Local Sentiment Aggregation (LSA)

The proposed LSA paradigm shifts the focus from individual aspects to a sentiment aggregation window. This window encompasses the target aspect and its $k$ nearest neighbors.

Architecture and Components

The LSA architecture integrates several key components to process these windows effectively:

Overview of the LSA model architecture, showing the integration of global context and aspect-focused features within a sentiment aggregation window.
  • Feature Variants: LSA can be implemented using different representation strategies, including Sentence Pair-based ($LSA_P$), Local Context-based ($LSA_T$), and Syntactical-based ($LSA_S$).
  • Differential Weighted Aggregation (DWA): Instead of simple averaging, LSA employs learnable weights to dynamically determine the influence of left and right neighbors. This allows the model to prioritize the most relevant neighboring signals.
  • Window Padding: To handle cases where an aspect has no neighbors, a "tripling" strategy is used, ensuring the target aspect remains the primary focus even when neighbors are absent.

Experimental Results

LSA was evaluated across six benchmarks, including SemEval (Laptop14, Restaurant14/15/16) and MAMS.

Performance and Robustness

When integrated with a DeBERTa-V3 backbone, LSA achieved state-of-the-art results. Notably, on the Restaurant14 dataset, an ensemble version ($LSA_E$) reached an accuracy of 91.43%.

Beyond raw accuracy, LSA demonstrates superior adversarial robustness. On the Laptop14-ARTS dataset—designed specifically to fool models with misleading context—the LSA-enhanced DeBERTa model improved accuracy from 67.71% to 74.47%. The aggregation window acts as a stabilizer, "smoothing out" adversarial noise by grounding predictions in the broader local sentiment context.

Discussion and Conclusion

LSA offers a computationally efficient alternative to complex graph-based models. By leveraging the inherent coherency of human language, it recovers implicit sentiments and resists local perturbations. While the model's advantage is less pronounced in mono-aspect sentences, its performance in multi-aspect scenarios sets a new standard for ABSC. Future work may involve exploring non-linear aggregation methods to better handle larger sentiment clusters.

Summary updated

Abstract

Aspect sentiment coherency is an intriguing yet underexplored topic in the field of aspect-based sentiment classification. This concept reflects the common pattern where adjacent aspects often share similar sentiments. Despite its prevalence, current studies have not fully recognized the potential of modeling aspect sentiment coherency, including its implications in adversarial defense. To model aspect sentiment coherency, we propose a novel local sentiment aggregation (LSA) paradigm based on constructing a differential-weighted sentiment aggregation window. We have rigorously evaluated our model through experiments, and the results affirm the proficiency of LSA in terms of aspect coherency prediction and aspect sentiment classification. For instance, it outperforms existing models and achieves state-of-the-art sentiment classification performance across five public datasets. Furthermore, we demonstrate the promising ability of LSA in ABSC adversarial defense, thanks to its sentiment coherency modeling. To encourage further exploration and application of this concept, we have made our code publicly accessible. This will provide researchers with a valuable tool to delve into sentiment coherency modeling in future research.
BibTeX
@inproceedings{yang2024modeling,
  author = {Heng Yang and Ke Li},
  title = {{Modeling Aspect Sentiment Coherency via Local Sentiment Aggregation}},
  booktitle = {EACL (Findings)},
  year = {2024},
}
Figure