Introduction
Sentiment analysis models typically operate on static datasets, failing to adapt as new products and domains emerge. Continual Learning (CL) addresses this by enabling models to learn tasks sequentially. However, CL faces two primary challenges: Catastrophic Forgetting (CF), where new knowledge overwrites old, and the Task-ID requirement, which necessitates knowing the domain source during inference. This paper introduces CLASSIC (Continual and Learning for Aspect Sentiment Classification), a framework designed for Domain Incremental Learning (DIL) in Aspect Sentiment Classification (ASC).
Methodology
CLASSIC utilizes a frozen BERT backbone augmented with Adapters—small, trainable layers that preserve core weights while learning task-specific features. The framework's novelty lies in its three contrastive learning objectives:
- Contrastive Ensemble Distillation (CED): Prevents forgetting by aligning the current model's outputs with an ensemble of previous "teacher" models using contrastive loss.
- Contrastive Knowledge Sharing (CKS): Employs a task-based self-attention mechanism to identify and pull relevant knowledge from similar previous domains.
- Contrastive Supervised Learning (CSC): Clusters embeddings with the same sentiment labels while separating different ones to refine internal representations.
Additionally, Task Masks (TMs) are employed to protect neurons associated with previous tasks, ensuring they remain unmodified during subsequent training phases.
Experimental Setup
The model was evaluated on a sequence of 19 ASC tasks sourced from four major datasets: HL5Domains, Liu3Domains, Ding9Domains, and SemEval14. This extensive benchmark covers diverse products ranging from electronics (e.g., Nokia 6610, Canon G3) to software (Norton) and services (Restaurants). The evaluation compared CLASSIC against 46 baselines, including state-of-the-art CL methods like HAT, EWC, and LAMOL.
Results and Discussion
CLASSIC achieves state-of-the-art performance across the 19-task sequence, demonstrating the effectiveness of the contrastive approach in a task-ID-free setting.
Key findings include:
- Superior Accuracy: CLASSIC achieved an average Accuracy of 0.9022 and a Macro-F1 of 0.8512, significantly outperforming LAMOL (0.8891 Accuracy).
- Effective Knowledge Transfer: The framework successfully achieved both positive forward transfer (leveraging old knowledge for new tasks) and backward transfer (refining old task performance through new learning).
- Robustness of Adapters: The study confirms that Adapters provide a more stable "workspace" for new knowledge compared to standard BERT fine-tuning, which often captures brittle, task-specific features.
Conclusion
CLASSIC proves that combining Adapters with contrastive learning objectives effectively mitigates catastrophic forgetting in domain-incremental settings. By treating the final model as a "universal" classifier, it eliminates the need for Task-IDs at inference, making it highly suitable for real-world sentiment analysis applications. Future work may extend this architecture to extraction-based tasks and other NLP domains like Named Entity Recognition.
Summary updated