Introduction
In social media analytics, a persistent challenge is the mismatch between training data and real-world deployment. While the "positive" class (e.g., posts about "politics") is often well-defined, the "negative" class—representing everything else—is an infinite, ever-changing sea of topics. This phenomenon is known as negative covariate shift: a classifier trained to distinguish politics from sports and movies may fail when confronted with posts about cooking or technology.
The CBS-L Methodology
To address this, the authors propose Center-Based Similarity Learning (CBS-L). Instead of working in the traditional Document Space (DS) of n-gram frequencies, CBS-L transforms documents into a space defined by their similarity to a positive class "center."
The pipeline involves:
- Feature Extraction: Documents are represented using tf-idf weighted unigrams, bigrams, and trigrams.
- Finding the Center: The algorithm computes a positive class center vector using the Rocchio method, which subtracts a weighted version of negative documents to "push" the center away from non-discriminative terms.
- Space Transformation: Documents are re-represented as vectors of similarity scores (e.g., Cosine, Gower, Lorentzian, Dice, and Jaccard) relative to the center.
- Learning: A standard binary SVM is trained on these similarity features.
By creating a "similarity bubble," the model effectively rejects any document that is "too different" from the center, regardless of whether that specific negative topic was encountered during training. As illustrated above, a linear boundary might misclassify unseen negative topics (triangles) that fall on the positive side of the hyperplane, whereas the similarity bubble correctly excludes them.
Experimental Evaluation
The method was evaluated using an Amazon review database covering 50 product types. To simulate negative covariate shift, models were trained on a subset of negative topics (varying from 10 to 40) and tested on a "Combined" set containing both seen and unseen negative topics.
Key Results
CBS-L demonstrated significant robustness compared to standard SVMs and One-Class SVMs (OSVM).
As shown in the results table, when only 10 negative topics were available for training, CBS-L achieved a combined F1-score of 0.481, nearly doubling the performance of a standard SVM (0.262) and a One-Class SVM (0.205). The most dramatic gains occurred in the "Out-of-training" category, where CBS-L maintained much higher precision and recall on topics the model had never encountered. Across all 50 topics, CBS-L reached an average F1-score of 0.598, compared to 0.439 for the baseline SVM.
Conclusion
CBS-L offers a robust solution for social media classification by focusing on the "wholeness" of the positive class rather than just looking for a line of separation. While it requires some negative data for center refinement and assumes a stable positive class, its ability to handle unknown negative topics makes it a powerful tool for real-world deployment where the "irrelevant" data is unpredictable.
Summary updated