Abstract
Aspect sentiment coherency is an intriguing yet underexplored topic in the field of aspect-based sentiment classification. This concept reflects the common pattern where adjacent aspects often share similar sentiments. Despite its prevalence, current studies have not fully recognized the potential of modeling aspect sentiment coherency, including its implications in adversarial defense. To model aspect sentiment coherency, we propose a novel local sentiment aggregation (LSA) paradigm based on constructing a differential-weighted sentiment aggregation window. We have rigorously evaluated our model through experiments, and the results affirm the proficiency of LSA in terms of aspect coherency prediction and aspect sentiment classification. For instance, it outperforms existing models and achieves state-of-the-art sentiment classification performance across five public datasets. Furthermore, we demonstrate the promising ability of LSA in ABSC adversarial defense, thanks to its sentiment coherency modeling.
Introduction: The Challenge of Aspect-Based Sentiment
In Aspect-Based Sentiment Classification (ABSC), the objective is to determine the sentiment toward specific entities within a sentence. For example, in the sentence "The screen is brilliant but the battery life is short," a model must identify "screen" as positive and "battery life" as negative.
Traditional approaches often treat these aspects as independent tasks or rely on complex, computationally expensive syntax trees to map linguistic dependencies. However, these methods frequently overlook a simple yet pervasive pattern in natural language: sentiment coherency. Humans often group similar sentiments together, creating "vibe" consistency across adjacent aspects.
The Concept of Aspect Sentiment Coherency
Sentiment coherency refers to the observation that adjacent aspects in a text (like "food" and "service" in a restaurant review) usually share the same sentiment polarity. The authors demonstrate that a vast majority of aspects in public datasets belong to "sentiment clusters" where neighbors share the same sentiment.
Existing models struggle to leverage this consistency, often remaining fragile to small adversarial changes. By modeling this coherency, we can create more robust classifiers that "borrow" sentiment signals from neighboring aspects when the local signal is ambiguous.
The Local Sentiment Aggregation (LSA) Framework
To address these limitations, we propose Local Sentiment Aggregation (LSA). Instead of analyzing an aspect in isolation, LSA constructs a "sentiment aggregation window" that includes the target aspect and its $k$ nearest neighbors (typically $k=1$).
Core Components
- Feature Learning: The model utilizes Pre-trained Language Models (PLMs) like BERT or DeBERTa. We introduce three variants: $LSA_P$ (standard sentence-pair), $LSA_T$ (local context weighting), and $LSA_S$ (syntax-aware weighting).
- Differential Weighted Aggregation (DWA): Recognizing that not all neighbors are equally informative, LSA uses learnable weights to dynamically determine the influence of left and right adjacent aspects on the target aspect's final representation.
- Window Padding: For single-aspect sentences, a "copy" strategy is used to maintain the window structure without introducing noise.
Key Results and Robustness
LSA achieves state-of-the-art (SOTA) results across five primary benchmark datasets (Laptop14, Restaurant14, 15, 16, and MAMS). Notably, the model excels in "Exact Match" (EM) tasks for predicting sentiment clusters, proving it effectively captures coherency.
Beyond accuracy, LSA provides a significant boost to adversarial defense. On the ARTS (Adversarial Robustness Test Set), LSA models showed massive improvements over baselines—for instance, jumping from a baseline BERT accuracy of ~64% to ~74% on Laptop14-ARTS. Modeling coherency acts as a "smoothing" mechanism; even if an attacker perturbs the words around one aspect, the stable sentiment of adjacent aspects helps the model maintain the correct prediction.
Conclusion
LSA offers a simple yet powerful paradigm for ABSC by leveraging the natural tendency of sentiment to cluster locally. By moving away from heavy computational dependencies like deep Graph Convolutional Networks (GCNs) and focusing on local aggregation, LSA provides a more efficient and robust path forward for sentiment analysis.
Summary updated