Abstract
Aspect-based sentiment analysis (ABSA) tries to predict the polarity of a given document with respect to a given aspect entity. While neural network architectures have been successful in predicting the overall polarity of sentences, aspect-specific sentiment analysis still remains as an open problem. In this paper, we propose a novel method for integrating aspect information into the neural model. More specifically, we incorporate aspect information into the neural model by modeling word-aspect relationships. Our novel model, Aspect Fusion LSTM (AF-LSTM) learns to attend based on associative relationships between sentence words and aspect which allows our model to adaptively focus on the correct words given an aspect term. This ameliorates the flaws of other state-of-the-art models that utilize naive concatenations to model word-aspect similarity. Instead, our model adopts circular convolution and circular correlation to model the similarity between aspect and words and elegantly incorporates this within a differentiable neural attention framework. Finally, our model is end-to-end differentiable and highly related to convolution-correlation (holographic like) memories. Our proposed neural model achieves state-of-the-art performance on benchmark datasets, outperforming ATAE-LSTM by 4%-5% on average across multiple datasets.
Introduction
In standard sentiment analysis, sentences are typically labeled with a single polarity. However, Aspect-Based Sentiment Analysis (ABSA) requires a more granular approach, identifying sentiment toward specific entities within a sentence. For instance, in the sentence "The food was delicious, but the service was terrible," the sentiment is positive for the "food" aspect but negative for "service."
Existing deep learning models, such as ATAE-LSTM, often handle this by concatenating aspect vectors to every word vector. This approach introduces several challenges: it burdens the LSTM and attention layers with learning relationships they weren't designed for, makes training difficult due to aspect-dominated input sequences, and leads to parameter bloat. The proposed AF-LSTM (Aspect Fusion LSTM) addresses these issues by replacing naive concatenation with a dedicated associative fusion layer.
Methods: The AF-LSTM Architecture
The core innovation of AF-LSTM is the introduction of a dedicated Association Layer between the LSTM and the Attention layer. This layer is designed to model the interaction between an aspect and a word as an associative memory problem.
Associative Memory Operators
Instead of concatenation, the model uses circular convolution or circular correlation to fuse the LSTM's hidden states ($h$) with the aspect embedding ($s$). Inspired by "Holographic Reduced Representations," these operators store and retrieve associations in a compressed format. These operations are highly efficient, computed in the frequency domain using Fast Fourier Transforms (FFT) with a complexity of $O(d \log d)$.
Parameterless Fusion
Unlike standard neural layers, this association layer is essentially parameterless. It does not add new weights but instead changes how existing information is combined. By handling the word-aspect relationship here, the subsequent attention layer can focus solely on the relative importance of the fused context words.
Experimental Results
The researchers evaluated AF-LSTM on standard benchmarks from SemEval 2014 and 2015, covering both Term Classification and Category Classification across Laptop and Restaurant datasets.
Performance and Efficiency
AF-LSTM (CONV) consistently outperformed all baselines, beating the previous state-of-the-art ATAE-LSTM by 4% to 8% on 3-way classification tasks. Notably, AF-LSTM achieves these results with significantly fewer parameters—approximately 810K compared to the 1.4M required by ATAE-LSTM. This demonstrates that specialized architectural choices for fusion are more effective than "brute force" concatenation.
Qualitative Insights
Qualitative analysis showed that AF-LSTM is superior at "switching" focus. When the aspect is "Appetizer," the model successfully ignores words related to "Service," and vice versa. This ability to decouple conflicting sentiments in the same sentence is a significant improvement over previous models. Interestingly, circular convolution (which is symmetric) generally performed better than circular correlation (asymmetric), suggesting that word-aspect relationships in ABSA are often mutual.
Conclusion
AF-LSTM introduces a robust and efficient method for aspect-based sentiment analysis by leveraging associative fusion. By separating the modeling of word-aspect relationships from the attention mechanism, the model achieves state-of-the-art performance with a smaller memory footprint. Future work may involve applying these associative fusion techniques to other query-context problems in NLP, such as Question Answering or Machine Reading Comprehension.
Summary updated