Abstract
Aspect-Category Sentiment Analysis (ACSA) involves identifying sentiment polarities for predefined categories within a text. Traditional multi-task learning (MTL) approaches often suffer from catastrophic forgetting when new categories are introduced incrementally. We propose CNE-net, a framework that treats category names as semantic inputs rather than abstract labels. By utilizing a shared encoder and decoder architecture, CNE-net maintains performance on existing categories while learning new ones, achieving state-of-the-art results on standard benchmarks.
Introduction
In real-world Natural Language Processing (NLP) applications, the set of relevant aspect categories is rarely static. For instance, a sentiment analyzer for the automotive industry may need to incorporate "battery life" as electric vehicles gain market share. Most existing ACSA models employ a Multi-Task Learning (MTL) strategy with a shared encoder but separate decoders for each category. When these models are updated with new categories—a process known as incremental learning—the shared encoder often shifts its representation, leading to a significant performance drop on previously learned categories, a phenomenon called catastrophic forgetting.
Methodology: The CNE-net Framework
The proposed Category Name Embedding network (CNE-net) addresses these issues by incorporating category names directly into the model's input. This allows the model to leverage the semantic meaning of categories to identify relevant linguistic cues within a sentence.
Architecture
The input to the BERT encoder is structured as: [CLS] sentence [SEP] category1 [SEP] category2 ... [SEP] categoryN. This design enables the model to understand the relationship between the sentence and specific categories (e.g., linking "too expensive" to the "price" category).
Unlike traditional models, CNE-net uses a single shared decoder. Since category information is embedded in the input features, the decoder learns a universal mapping from these features to sentiment polarities. The authors explored several attention-based decoder variants, identifying CNE-net-SEP-sent.-att. as the most effective. This variant employs a two-step attention mechanism: first, the category's separator token ([SEP]) attends to the sentence; then, the resulting representation attends back to the category name to generate highly specific features.
Experimental Evaluation
The framework was evaluated on two benchmark datasets: SemEval-2014 Task 4 (restaurant reviews) and SentiHood (location-based reviews). To test incremental learning capabilities, the authors created SemEval14-Task-inc and Sentihood-inc, where categories are partitioned into "source" (previously learned) and "target" (newly introduced) sets.
Results and Discussion
CNE-net achieved state-of-the-art (SOTA) performance across both benchmarks. On the SemEval-14 dataset, it reached a sentiment accuracy of 87.1% for 4-way classification, outperforming existing models like QA-B and standard MTL.
Incremental Learning Stability
The primary strength of CNE-net is its resilience to catastrophic forgetting. In incremental tests, standard MTL models experienced drastic performance declines on source categories. In contrast, CNE-net maintained stable performance on old categories even after training on new ones.
The results demonstrate that sharing the decoder is critical for incremental learning. Because the decoder weights are universal, fine-tuning on new categories refines the general sentiment-mapping ability rather than overwriting category-specific logic. Furthermore, the model shows a strong ability to capture implicit sentiment, correctly associating descriptive phrases with categories even when the category name is absent from the review text.
Conclusion
CNE-net provides a robust solution for incremental ACSA by treating categories as semantic entities. By sharing both encoders and decoders, the model effectively mitigates catastrophic forgetting. Future work may explore the zero-shot learning potential of this architecture, as the model's understanding of category names could allow it to predict sentiments for entirely unseen categories.
Summary updated