Aspect-Context Interactive Attention Representation for Aspect-Level Sentiment Classification

Zhuojia Wu, Yang Li, Jian Liao, Deyu Li, Xiaoli Li, Suge Wang

Introduction

Aspect-Level Sentiment Classification (ALSC) represents a significant advancement in natural language processing, moving beyond global sentiment analysis to identify the polarity of specific features within a text. For instance, in a review stating, "The menu is limited, but the dishes are excellent," an ALSC model must distinguish the negative sentiment associated with the "menu" from the positive sentiment directed at the "dishes."

Traditionally, this task has been dominated by Long Short-Term Memory (LSTM) networks. However, these models suffer from two primary limitations: sequential processing bottlenecks that hinder parallelization and isolated modeling where context is processed before the aspect is fully considered. This often results in a failure to capture how specific aspect words should influence the interpretation of the surrounding context.

The MultiACIA Architecture

To address these challenges, researchers have proposed MultiACIA (Multi-layer Aspect-Context Interactive Attention). This architecture departs from recurrence, relying instead on a purely attentive, interactive approach inspired by the Transformer.

The core of the model is the Aspect-Context Interactive Attention (ACIA) mechanism. Unlike standard attention that filters context through a pre-computed aspect vector, ACIA calculates a pair-wise matching matrix between every word in the aspect and every word in the context simultaneously.

The Aspect-Context Interactive Attention (ACIA) mechanism.

As illustrated in the figure above, the ACIA block generates a matching matrix $M$ from the aspect ($T$) and context ($S$). By applying column-wise and row-wise softmax operations, the model derives attention weights ($\alpha$ and $\beta$) that highlight the most relevant interactions. This allows the model to "weight" the importance of context words based on the aspect and vice versa.

MultiACIA further enhances this representation through:

  • Multi-Head Structure: Utilizing multiple attention heads to capture diverse linguistic relationships, such as syntactic dependencies and specific descriptive adjectives.
  • Deep Stacking: The model stacks four interactive layers, employing residual connections and layer normalization to maintain stability and prevent performance degradation in deep networks.

Experimental Evaluation

The MultiACIA model was evaluated against 13 state-of-the-art methods using three benchmark datasets:

  1. SemEval-2014 Task 4 (Restaurant): Characterized by a high frequency of multiple aspects per sentence (over 50% co-occurrence).
  2. SemEval-2014 Task 4 (Laptop): Contains more implicit sentiment expressions.
  3. Twitter Dataset: Features short, informal social media posts.

Key Results

MultiACIA demonstrated superior performance across all metrics. In the Restaurant dataset, it achieved a state-of-the-art accuracy of 82.59% and a Macro-F1 score of 72.13%. On average, the model provided a 4.52% boost in accuracy over standard LSTM baselines across the three datasets.

Beyond accuracy, the model offers significant efficiency gains. By eliminating recurrent connections, MultiACIA supports full parallelization, drastically reducing training times compared to traditional LSTM-based architectures.

Discussion and Future Directions

A critical insight from this research is that the aspect should guide the entire representation process rather than acting as a final filter. MultiACIA successfully learns to ignore irrelevant tokens (like "the" or "is") and focus on opinion words that map directly to the target aspect. It also handles multi-word aspects (e.g., "chicken with black bean sauce") more effectively by identifying the "core" noun.

However, the transition to pure attention results in some loss of inherent word order information. While positional encodings mitigate this, future work may involve incorporating semantic dependency information—explicitly modeling the grammatical relationships between words—to further refine the link between adjectives and their corresponding nouns in complex sentences.

Conclusion

MultiACIA represents a shift toward more interactive and parallelizable models for fine-grained sentiment analysis. By modeling the relationship between aspect and context simultaneously through deep, multi-head attention, it sets a new benchmark for accuracy and efficiency in the field of Aspect-Level Sentiment Classification.

Summary updated

BibTeX
@article{wu2020aspect,
  author = {Zhuojia Wu and Yang Li and Jian Liao and Deyu Li and Xiaoli Li and Suge Wang},
  title = {{Aspect-Context Interactive Attention Representation for Aspect-Level Sentiment Classification}},
  journal = {IEEE Access},
  year = {2020},
  doi = {10.1109/access.2020.2972697},
}
Figure