Abstract
Social news aggregator services generate readers’ subjective reactions to news opinion articles. Can we use those as a resource to classify articles as liberal or conservative, even without knowing the self-identified political leaning of most users? We applied three semi-supervised learning methods that propagate classifications of political news articles and users as conservative or liberal, based on the assumption that liberal users will vote for liberal articles more often, and similarly for conservative users and articles. Starting from a few labeled articles and users, the algorithms propagate political leaning labels to the entire graph. In cross-validation, the best algorithm achieved 99.6% accuracy on held-out users and 96.3% accuracy on held-out articles. Adding social data such as users’ friendship or text features such as cosine similarity did not improve accuracy. The propagation algorithms, using the subjective liking data from users, also performed better than an SVM based text classifier, which achieved 92.0% accuracy on articles.
Introduction
In the polarized landscape of U.S. politics, understanding the "leaning" of content and its consumers is vital for building balanced recommendation engines and analyzing social discourse. However, two major hurdles exist: first, most users do not explicitly state their political affiliation; second, political stance in text is often subtle, making it difficult for traditional natural language processing (NLP) to distinguish between reporting and advocacy. This research explores whether we can bypass textual ambiguity by analyzing voting patterns ("Diggs") on social news platforms.
Methodology: Graph-Based Label Propagation
The problem is modeled as a bipartite graph where nodes represent Users and Stories, and edges represent Votes. By identifying a small set of "seed" liberal and conservative nodes, labels can be propagated across the network.
Three semi-supervised learning (SSL) algorithms were evaluated:
- Random Walk with Restart (RWR): Simulates a walker moving through the graph, occasionally returning to seed nodes.
- Local Consistency Global Consistency (LCGC): An optimization approach assuming nearby nodes share labels while respecting initial seeds.
- Absorbing Random Walk (ARW): Treats labeled nodes as "sinks" that absorb probability. This method was the top performer.
Dataset and Ground Truth
The study utilized a massive crawl of Digg.com data (2009–2010), comprising 84,433 popular stories and 74,844 frequent users, connected by over 5.2 million votes. Ground truth labels were established through news reports (e.g., "Digg Patriots"), manual coding of 220 users, third-party blog classifications (1,635 blogs), and 1,000 stories manually coded via Amazon Mechanical Turk with unanimous agreement.
Key Results and Insights
The Absorbing Random Walk (ARW) algorithm achieved a peak overall accuracy of 97.3%, significantly outperforming a standard Support Vector Machine (SVM) text classifier (92.0%).
A surprising finding was that adding "social" features (friendship links) or "textual" features (cosine similarity) actually decreased accuracy. As shown in the parameter sensitivity analysis (Figure 5), tuning weights for different edge types (source, user, story) reveals that the voting signal itself is the most robust indicator. Friends may share hobbies but not politics, and articles on the same topic (e.g., healthcare) may have high text similarity despite opposing viewpoints.
Limitations and Future Work
While highly accurate for "clear" cases, accuracy drops to ~72-75% when including "gray" items where human raters disagree. Additionally, the "cold start" problem remains: the algorithm requires votes to function, unlike text-based classifiers. Future work may involve hybrid models where graph propagation generates high-quality labels to train more robust text-based classifiers.
Summary updated