Abstract
Aspect sentiment classification, a challenging task in sentiment analysis, has been attracting more and more attention in recent years. In this paper, we highlight the need for incorporating the importance degrees of both words and clauses inside a sentence and propose a hierarchical network with both word-level and clause-level attentions to aspect sentiment classification. Specifically, we first adopt sentence-level discourse segmentation to segment a sentence into several clauses. Then, we leverage multiple Bi-directional LSTM layers to encode all clauses and propose a word-level attention layer to capture the importance degrees of words in each clause. Third and finally, we leverage another Bi-directional LSTM layer to encode the outputs from the former layers and propose a clause-level attention layer to capture the importance degrees of all the clauses inside a sentence. Experimental results on the laptop and restaurant datasets from SemEval-2015 demonstrate the effectiveness of our proposed approach to aspect sentiment classification.
Introduction
Sentiment analysis has evolved from document-level polarity detection to fine-grained Aspect Sentiment Classification (ASC). ASC seeks to determine the sentiment expressed toward specific entities or attributes within a sentence. A primary challenge in this task is the complexity of natural language, where a single sentence often contains multiple sentiments tied to different aspects. For instance, in a restaurant review, a user might praise the food while criticizing the seating space.
Traditional neural networks often treat sentences as flat sequences of words, which can lead to "noise" where strong sentiment words for one aspect interfere with the classification of another. While word-level attention mechanisms have improved performance, they often overlook the structural reality that sentences are composed of clauses—distinct units of meaning that typically house a single sentiment.
Methodology
The proposed Hierarchical Aspect-specific Attention Network aligns neural architecture with linguistic structure. The model operates in three main stages:
- Discourse Segmentation: Using Rhetorical Structure Theory (RST), the model segments a sentence into Elementary Discourse Units (EDUs), which function as clauses.
- Word-level Attention: For each clause, a Bi-directional LSTM (Bi-LSTM) encodes the words. An attention mechanism then weights these words based on their relevance to the specific aspect being analyzed. To supplement training, the researchers used Predictive Text Embedding (PTE) to build large-scale word representations from Amazon review data.
- Clause-level Attention: This hierarchical step takes the summarized vectors of each clause and processes them through another Bi-LSTM. A second attention layer determines which clauses are most important for the aspect. For example, a clause about "delicious food" is weighted heavily for the "FOOD" aspect, while clauses about "wait time" or "ambience" are suppressed.
Results and Analysis
The model was evaluated on standard benchmarks from the SemEval-2015 Task 12 (Restaurant and Laptop domains). The hierarchical approach consistently outperformed state-of-the-art baselines, achieving 80.9% accuracy on the Restaurant dataset, compared to 78.7% for the previous top system and 75.2% for standard attention models.
An ablation study confirmed that the combination of word-level and clause-level attention is essential. Visualization of the attention weights proves that the model successfully filters out irrelevant clauses. When analyzing "FOOD#QUALITY," the model effectively "shuts off" parts of the sentence discussing unrelated aspects like ambience, as shown in the heatmap above.
Conclusion
By incorporating both word-level and clause-level importance, this hierarchical network provides a more robust framework for aspect-based sentiment analysis. The model demonstrates that aligning neural structures with the linguistic reality of clauses significantly reduces noise in complex sentences. Future research may explore deeper discourse analysis to understand how logical relationships between clauses (e.g., contrast or concession) impact sentiment polarity.
Summary updated