Abstract
Sentiment analysis is a costly yet necessary task for enterprises to study the opinions of their customers to improve their products and to determine optimal marketing strategies. Due to the existence of a wide range of domains across different products and services, cross-domain sentiment analysis methods have received significant attention. These methods mitigate the domain gap between different applications by training cross-domain generalizable classifiers which relax the need for data annotation for each domain. We develop a domain adaptation method which induces large margins between data representations that belong to different classes in an embedding space. This embedding space is trained to be domain-agnostic by matching the data distributions across the domains. Large interclass margins in the source domain help to reduce the effect of "domain shift" in the target domain. Theoretical and empirical analysis are provided to demonstrate that the proposed method is effective.
Introduction
Sentiment analysis is a cornerstone of modern business intelligence, yet it faces a significant "vocabulary problem." A book review might use terms like "fascinating" or "boring," whereas a kitchen appliance review might focus on "durability" or "efficiency." When an AI model trained on one domain (e.g., Books) encounters another (e.g., Kitchen), it hits a domain gap. The specific linguistic cues change, causing the model's internal representation of sentiment to shift.
Most existing solutions attempt to "align" these domains so their distributions overlap. However, if the decision boundaries between positive and negative classes are too narrow, even a minor shift in the target domain can lead to misclassification. Researchers at USC have addressed this by developing SAIM² (Sentiment Analysis using Increased-Margin Models), a method that prioritizes "breathing room"—or large margins—between classes to ensure robustness.
Methodology: The SAIM² Framework
The SAIM² framework follows a structured pipeline designed to create a robust, domain-agnostic embedding space:
- Initial Training: An encoder and classifier are pre-trained on a labeled source domain (e.g., Books).
- Prototypical Distribution Estimation: Rather than relying on raw data points, the model estimates the "shape" of the data in the embedding space using a Gaussian Mixture Model (GMM). By leveraging existing labels, the mean and variance for each class are calculated using efficient, closed-form solutions.
- Pseudo-Dataset Generation: Random samples are drawn from the GMM. To ensure high quality, only "confident" samples—those furthest from the decision boundary—are retained. This creates a "clean" source representation with intentionally widened gaps between classes.
- Domain Adaptation via SWD: The target domain is aligned with this high-margin pseudo-dataset using Sliced Wasserstein Distance (SWD). SWD is computationally efficient and provides a smooth gradient for learning, facilitating alignment even when initial distributions are disparate.
Experimental Evaluation
The effectiveness of SAIM² was evaluated using the standard Amazon Reviews benchmark dataset, covering four domains: Books (B), DVD (D), Electronics (E), and Kitchen (K). The researchers conducted 12 pairwise cross-domain tasks (e.g., Books $\rightarrow$ Kitchen). To demonstrate versatility, the method was tested across different feature extraction techniques, including traditional $tf-idf$ (bag-of-words) and modern BERT embeddings.
Results and Discussion
The core insight of SAIM² is that alignment alone is insufficient; the "margin" is king. By forcing source classes to maintain physical space in the embedding map, the model creates a buffer. When the target domain shifts due to vocabulary changes, it is less likely to drift across the decision boundary.
Performance Benchmarks
On $tf-idf$ features, SAIM² consistently outperformed state-of-the-art methods like Domain Separation Networks (DSN) and Central Moment Discrepancy (CMD). For instance, in the DVD $\rightarrow$ Kitchen task, SAIM² achieved 87.3% accuracy, a significant improvement over the source-only baseline of 79.5%. When utilizing BERT features, SAIM² remained highly competitive, often matching or exceeding specialized models like BERT-DA.
Robustness and Theoretical Foundation
Beyond raw accuracy, SAIM² demonstrated remarkable stability under data imbalance (e.g., 90% positive vs. 10% negative reviews), where many competing models degrade. This empirical success is backed by theoretical analysis based on PAC-learning theory, proving that the algorithm minimizes the upper bound of error in the target domain.
Conclusion
SAIM² presents a significant step forward in unsupervised domain adaptation by emphasizing the importance of interclass margins. While the method assumes some degree of overlap between domains, its ability to "remember" the essence of source data via GMMs makes it both efficient and robust. Future work may explore "source-private" adaptation to further enhance data privacy. As Large Language Models continue to evolve, specialized methods like SAIM² remain vital for building efficient, high-performance models in resource-constrained environments.
Summary updated