Abstract
Sentiment classification is inherently domain-dependent, as the polarity of words can shift significantly across different contexts. This paper presents a novel "completely-shared" neural network architecture that dynamically adjusts word representations using domain-aware embedding and attention mechanisms. By integrating a shared domain detector with a sentiment classifier, the model achieves state-of-the-art performance across 16 diverse domains, outperforming even fine-tuned BERT models while maintaining high parameter efficiency.
Introduction
Determining the sentiment of a text—whether it is positive or negative—is a task heavily influenced by the domain of the content. A phrase like "easy to guess" might be a criticism in a movie review but a positive attribute in a different context. Traditional models often struggle with this variability because they utilize static word embeddings. While "Share-Private" models attempt to solve this by using separate parameters for different domains, they are often computationally expensive and difficult to scale.
This research introduces a model where all parameters are shared across domains, but internal representations are made "domain-aware." This approach allows the network to dynamically "re-flavor" word meanings based on the detected domain, leading to more accurate sentiment predictions without the overhead of domain-specific private layers.
Methodology
The proposed architecture consists of three integrated components designed to inject domain context into the sentiment analysis pipeline:
- Domain Feature Extraction: The model employs a Bidirectional LSTM (BiLSTM) to classify the input text into its respective domain (e.g., "Electronics" or "Books"). The hidden states from this BiLSTM provide a vector representation of the domain context.
- Domain-Aware Embedding: These domain features are concatenated with general-purpose word embeddings (such as GloVe). This creates a domain-aware embedding where the vector for a specific word is modified by the detected domain context.
- Domain-Aware Attention: A second BiLSTM processes these embeddings to extract sentiment features. A domain-aware attention mechanism then uses the overall domain representation as a "query" to determine which words are most significant for sentiment in that specific context. For instance, the model learns to prioritize words like "infantile" as negative signals in book reviews while ignoring them in baby product reviews.
The entire system is trained using a joint loss function, optimizing for both domain classification accuracy and sentiment classification accuracy simultaneously.
Experimental Setup
The model was evaluated on a standard benchmark dataset comprising reviews from 16 distinct domains, including 14 product categories (e.g., Apparel, Camera, Health, Toys) and 2 movie review datasets (IMDB and MR). The data was split into 70% for training, 10% for development, and 20% for testing. The primary goal was to assess the model's accuracy, efficiency, and ability to generalize to unseen domains.
Results and Discussion
The experimental results highlight the effectiveness of the domain-aware approach:
- Superior Accuracy: The model achieved an average accuracy of 90.1% across all 16 domains, setting a new state-of-the-art for this benchmark.
- Outperforming Large Language Models: The proposed architecture outperformed a fine-tuned BERT-base model (88.1%) and specialized multi-task models like DSR-at (87.9%).
- Cross-Domain Generalization: In "leave-one-out" tests, where the model was tested on a domain not seen during training, it maintained a high average accuracy of 87.9%, demonstrating strong generalizability.
- Efficiency and Transferability: Because the model is completely shared, it requires significantly fewer parameters than traditional multi-task models. Furthermore, the domain feature extractor proved to be transferable, enhancing performance even when applied to domains outside the initial training set.
Conclusion
This paper demonstrates that a completely-shared architecture, when augmented with domain-aware embedding and attention, can effectively handle the challenges of multi-domain sentiment classification. The model provides a parameter-efficient alternative to complex "Share-Private" or large-scale transformer models while delivering superior accuracy. Future research will focus on applying these domain-aware techniques to more granular sentiment analysis and integrating them more deeply with modern transformer architectures.
Summary updated