Abstract
This paper introduces a semi-supervised learning approach for blog classification that exploits the inherent structure of blogs, where multiple entries share a single authorial "voice." By training thousands of sub-classifiers to distinguish between pairs of unlabeled blogs, the authors generate a rich feature set that significantly enhances classification accuracy, especially in scenarios where labeled training data is scarce.
Introduction
Classifying blogs to identify an author's gender, age, or interests is a cornerstone of modern social media analytics. However, traditional supervised learning is often constrained by the high cost and time required to produce manually labeled examples. While unlabeled blog data is abundant, standard text classification methods typically treat a blog as a single block of text, failing to capitalize on the unique structural property that one blog contains many entries written by the same person. This paper addresses how to use this inherent structure to learn writing styles and topics from unlabeled data to improve classification performance.
Methodology
The proposed method utilizes Alternating Structure Optimization (ASO) through the creation of Sub-Classifiers.
- Sub-Classifier Training: Instead of predicting a standard label, the system selects two random unlabeled blogs (Blog A and Blog B). Entries in Blog A are treated as positive examples, and entries in Blog B as negative examples.
- Learning Style and Topic: A linear classifier is trained to distinguish between these two specific blogs. This process forces the model to identify subtle differences in writing style, vocabulary, and topics.
- Feature Expansion: This process is repeated thousands of times ($K=1,000$ or $10,000$). For any new blog entry, the outputs of these $K$ sub-classifiers are concatenated with the original "bag-of-words" features.
- Main Classifier: Finally, a standard Support Vector Machine (SVM) is trained on the limited labeled data using this augmented feature vector.
Essentially, the sub-classifiers act as a "similarity map," describing a new blog entry by how much it "sounds like" thousands of other reference blogs.
Experimental Evaluation
The researchers evaluated their approach using labeled data from Yahoo! Blogs (2,112 for gender, 1,314 for age) and a large pool of unlabeled blogs from Livedoor.
Key Results
The proposed method consistently outperformed the baseline across both gender and age identification tasks.
- Low-Data Efficiency: The improvement was most dramatic when labeled training data was very small (e.g., 100 examples), proving the method's efficiency in data-sparse environments.
- Diminishing Returns: Interestingly, increasing the number of sub-classifiers from 1,000 to 10,000 did not always yield better results. In age classification, the 10,000-model version performed worse on hard-to-distinguish groups (like 20s vs. 30s), likely because the unlabeled features began to introduce noise.
Discussion and Future Work
The "automatic labeling" aspect of this method is particularly elegant, as it allows for the generation of infinite labeled pairs for sub-classifiers without human intervention. Furthermore, the independent training of sub-classifiers makes the process highly parallelizable.
Future work may explore:
- Strategic Selection: Choosing "representative" blogs for sub-classifiers rather than random selection.
- Influence Weighting: Tuning the impact of unlabeled data to prevent noise in high-$K$ scenarios.
- Domain Adaptation: Handling distribution variances between different blog service providers.
- Advanced Features: Integrating n-grams or stylistic markers like emoticons to further capture authorial nuance.
Summary updated