Introduction
Personalized sentiment analysis models typically rely on User and Product (UP) metadata to capture individual rating biases and product-specific nuances. While this "In-Domain" (ID) information significantly boosts performance, it creates a fragility: models often fail when encountering new or anonymous users (Out-of-Domain or OOD). This research addresses this gap by proposing a method to learn robust, domain-invariant textual representations that remain effective even when metadata is absent.
The core issue is that models tend to become "lazy," over-relying on domain-specific features ($F_s$) from metadata while neglecting the rich domain-invariant features ($F_i$) in the text. As illustrated in the figure below, the goal is to align these representations so the model can generalize to the "Generalized Domain" (GD) where metadata is unavailable.
Methodology: GSwitch and Switch Knowledge Distillation
The authors introduce two key components: the GSwitch Module and Switch Knowledge Distillation (SKD).
The GSwitch Module
GSwitch is a flexible architectural component that can be integrated into various text encoders (e.g., BERT, CNNs). It functions as a toggle with two states:
- Switch ON: Fuses textual features with UP metadata (using learned weights and biases) to create an "enhanced" representation.
- Switch OFF: Masks the metadata by setting it to zero, forcing the encoder to generate a representation based solely on the review text.
Switch Knowledge Distillation (SKD)
To ensure the text-only representation is as informative as the metadata-enhanced one, the authors employ SKD during training. For every review, the model generates two sets of logits: one with the switch ON (ID logits) and one with it OFF (GD logits).
As shown in the training workflow diagram, the "Switch ON" state acts as a teacher, while the "Switch OFF" state acts as a student. By minimizing the bidirectional Kullback–Leibler (KL) divergence between these logits (KD Loss), the model is forced to distill the insights gained from the metadata into the text-only encoder. This process ensures that the domain-invariant features ($F_i$) are robustly captured.
Experimental Evaluation
The researchers evaluated their approach on three benchmark datasets: IMDB, Yelp-2013, and Yelp-2014. To simulate real-world OOD scenarios, they masked user and product identifiers during the testing phase.
Key Results
- Superior OOD Performance: The proposed method consistently outperformed baseline models when metadata was missing. On the Yelp-2014 dataset, the GSwitch-BERT-DG model achieved an F1 score of 66.42, a notable improvement over the standard BERT's 65.60.
- Stability in ID Scenarios: Crucially, the model did not sacrifice accuracy for known users. In the "Switch ON" state, it remained competitive with state-of-the-art personalized models.
- Architectural Versatility: The GSwitch module proved effective across different base architectures, including CNNs and LSTMs, highlighting its broad applicability.
Conclusion
By introducing the GSwitch module and the SKD training paradigm, this research provides a robust solution for personalized sentiment analysis in the face of missing metadata. The method successfully prevents models from over-relying on metadata shortcuts, ensuring high performance for both identified and anonymous users. Future research may explore the application of GSwitch to other NLP tasks where user context is intermittent, such as dialogue systems or intent recognition.
Summary updated