“All I know about politics is what I read in Twitter”: Weakly Supervised Models for Extracting Politicians’ Stances From Twitter

Kristen Johnson, Dan GoldwasserCOLING 2016

TL;DR

This paper introduces a weakly supervised method to infer politicians' stances on political issues and their agreement patterns by analyzing their Twitter activity. By combining content, temporal, and framing features within a Probabilistic Soft Logic framework, they achieve high accuracy in predicting stances and agreement, outperforming traditional supervised methods.

What Problem Does This Paper Address?

The rise of social media, particularly Twitter, has made it a crucial platform for politicians to communicate their beliefs and stances on current events. However, the brevity of tweets and the need for careful messaging present challenges for accurately understanding a politician's true position. Previous work often focused on individual tweets in a supervised manner, failing to capture the dynamic and contextual nature of political discourse on Twitter. This paper addresses the need for a more holistic approach that considers a politician's overall Twitter behavior, including their framing of issues and temporal activity patterns, to infer their stances and identify alliances or rivalries with other politicians.

Methods

The authors propose a weakly supervised approach that leverages Probabilistic Soft Logic (PSL) to build global models from local, noisy classifiers. The core idea is to combine various signals from Twitter data to infer politicians' stances and agreement patterns.

  • Weakly Supervised Local Models: They first develop local models that extract features from tweets:
  • Issue Identification: Keyword-based heuristics to determine which issues a politician is tweeting about.
  • Sentiment Analysis: Using OpinionFinder to classify tweets as positive, negative, or neutral, with party majority sentiment used as a fallback.
  • Content Agreement: Measuring word similarity in tweets to infer potential agreement.
  • Temporal Activity Patterns: Identifying when politicians tweet about the same issue to infer agreement.
  • Political Framing: Analyzing keywords to categorize how politicians frame issues, reflecting their contextualization strategies.
  • Temporal Framing Patterns: Combining temporal activity and framing to capture event-driven communication.
  • Global PSL Model: These local models provide weak supervision for a global PSL model. PSL allows for specifying rules over relational data, enabling the model to learn weights and infer soft truth assignments (0-1) for target predicates like PRO(Politician, Issue) (stance) and SAMESTANCE(Politician1, Politician2) (agreement). The model iteratively builds upon itself, starting with party line assumptions and progressively incorporating more complex Twitter behavioral features.

Datasets

The study utilizes a dataset of 99,161 tweets from 32 prominent U.S. politicians (16 Republicans and 16 Democrats) collected during the 2016 U.S. presidential election. These tweets cover 16 distinct political issues, identified using questions from ISideWith.com. The stances for evaluation were manually annotated using online searches and voting records when unavailable on the site.

Key Insights and Ideas

  • Beyond Individual Tweets: The paper strongly argues that a politician's overall Twitter behavior, not just individual tweets, is crucial for accurately inferring their stances. This includes when they tweet and how they frame issues.
  • The Power of Weak Supervision: By using weakly supervised local models to bootstrap a global PSL model, the approach can generalize to new issues and politicians without requiring extensive labeled data for every scenario.
  • Framing as a Stance Indicator: The way politicians frame issues (e.g., as a matter of safety vs. freedom) is a significant indicator of their underlying stance, even when explicit sentiment is neutral.
  • Temporal Patterns Reveal Alliances: Synchronized tweeting on specific issues can signal agreement or disagreement, even if the explicit content is not directly comparable.
  • Party Lines as a Starting Point, Not an End: While party affiliation provides a strong initial bias, incorporating behavioral features allows the model to identify nuances and exceptions to party-line voting.

Key Results

The proposed PSL models (M1, M2, M3) consistently outperformed a supervised baseline (SVM) and a local baseline (LB) across most issues.

  • Overall Accuracy: Model M3, which incorporated all behavioral features, achieved the highest overall accuracy for stance prediction (89.84%) and agreement prediction (91.49%).
  • Improvement Over Baselines: M3 improved stance prediction accuracy by 11.16% over LB and agreement prediction by 24.68% over LB.
  • Issue-Specific Performance: For many issues, M3 achieved near-perfect accuracy (e.g., 100% for ACA stance and agreement). The paper notes that for some issues (like Guns, Iran, TPP), party information alone was insufficient, and behavioral features were critical for improvement.
  • Qualitative Findings: The paper highlights how temporal patterns and framing differences can reveal subtle distinctions in politicians' stances and agreements, even when party affiliation might suggest otherwise.

Limitations and Future Work

  • Limited Scope: The study focused on a small set of politicians and issues. The authors suggest that the approach can be extended to handle more politicians, issues, or even different countries by incorporating appropriate domain knowledge.
  • Data Availability: The authors note that their dataset had few "@" mentions or retweets, limiting their ability to analyze explicit conversation structures.
  • Future Directions: The authors propose adapting the framework for different countries, incorporating other forms of political discourse (e.g., parliamentary votes), and exploring more complex temporal and framing patterns. They also mention that jointly modeling stance and agreement can improve performance but leads to more complex models and slower inference.

Summary updated

BibTeX
@inproceedings{johnson2016all,
  author = {Kristen Johnson and Dan Goldwasser},
  title = {{“All I know about politics is what I read in Twitter”: Weakly Supervised Models for Extracting Politicians’ Stances From Twitter}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2016},
}

Loading related content...

Figure