Introduction
Traditional sentiment analysis typically focuses on absolute polarity—classifying a single text as positive, negative, or neutral. However, this approach often struggles with domain shifts; a model trained on restaurant reviews might learn that "delicious" is a strong positive signal but fail when applied to electronics reviews where "reliable" is the key descriptor.
This paper introduces the "same sentiment classification problem": a task where a model must determine if two texts share the same sentiment polarity without necessarily identifying what that sentiment is. By reframing sentiment analysis as a pairwise similarity task, the authors hypothesize that "sameness" is a more universal trait that relies less on topic-specific vocabulary, making models more robust when moving between different subjects.
Methodology
The researchers adapted the standard sequence pair classification approach using the BERT (Bidirectional Encoder Representations from Transformers) architecture. Specifically, they utilized BERT-base-uncased with a custom classification head consisting of a dropout layer followed by a dense layer with a sigmoid activation.
Instead of a single sentence, the model receives two reviews separated by a special token. The model then uses the representation of the [CLS] token to predict a binary outcome: "same sentiment" or "different sentiment." This approach was compared against traditional machine learning baselines (SVM and Logistic Regression using TF-IDF/Doc2Vec) and Siamese Bi-LSTM networks using GloVe embeddings.
Experimental Setup
The study primarily utilized the Yelp Business Review Dataset, which contains roughly 6.7 million reviews across 22 main business categories. Star ratings (1–5) were converted into binary labels: "good" (4–5 stars) and "bad" (1–3 stars).
The core of the data preparation was the pairing strategy: random pairs of reviews were matched about the same business. This ensures the model cannot rely on the topic to find a match; it must look specifically at the sentiment expressed about that topic. The authors found that increasing the sequence length to 512 tokens was necessary to capture enough nuance in longer business reviews.
Key Insights and Results
The best BERT-based model achieved an overall accuracy of 89.05% across all categories. One of the most significant findings was the "Good-Good" bias: the model was significantly better at identifying two positive reviews as "same" (91.2% accuracy) than identifying two negative reviews as "same" (84.6%). This suggests that linguistic markers of "goodness" are more consistent than the varied ways people express "badness."
Accuracy varied by topic, with "Financial Services" hitting a high of 95.05% and "Arts & Entertainment" at a lower 83.82%. Despite these variations, the model maintained high accuracy even when training and testing sets were from completely different categories, supporting the idea of topic agnosticism in sentiment agreement.
Conclusion and Future Work
Reframing sentiment as a similarity task allows for strong cross-domain generalization. While BERT outperformed Siamese Bi-LSTM and traditional ML models, the approach is computationally intensive. Future work should explore lighter transformer variants like DistilBERT or ALBERT to reduce overhead. Additionally, investigating whether this "sameness" approach works for fine-grained sentiment (e.g., distinguishing between "okay" and "amazing") remains an open area for research.
Summary updated