Modeling Ideological Agenda Setting and Framing in Polarized Online Groups with Graph Neural Networks and Structured Sparsity

Valentin Hofmann, Janet B. Pierrehumbert, Hinrich Schütze

Abstract

The increasing polarization of online political discourse calls for computational tools that are able to automatically detect and monitor ideological divides in social media. Here, we introduce a minimally supervised method that directly leverages the network structure of online discussion forums, specifically Reddit, to detect polarized concepts. We model polarization along the dimensions of agenda setting and framing, drawing upon insights from moral psychology. The architecture we propose, SLAP4SLIP, combines graph neural networks with structured sparsity and results in representations for concepts and subreddits that capture phenomena such as ideological radicalization and subreddit hijacking. We also create a new dataset of political discourse covering 12 years and more than 600 online groups with different ideologies.

Introduction

Detecting ideological divides in online discourse is a major challenge for computational social science. Most existing NLP methods for measuring polarization rely on "gold standard" labels—researchers must know in advance which texts are "left-wing" or "right-wing." However, in the wild west of social media, these labels are rarely available and expensive to produce.

This work addresses this gap by asking: Can we identify what people are polarized about just by looking at who they talk to? We leverage the principle of homophily—the idea that people with similar views tend to cluster together—to turn a social network's structure into a signal for linguistic polarization.

Methodology: The SLAP4SLIP Framework

We introduce SLAP4SLIP (Sparse LAnguage Properties for Social LInk Prediction). The core idea is to use a Graph Auto-Encoder (GAE) to predict the links between subreddits based solely on how they use specific words.

Visualization of the Reddit Politosphere network colored by salience (left) and framing (right) of polarized concepts.

The model analyzes polarization through two primary lenses:

  1. Salience (Agenda Setting): This measures how often a concept is mentioned. For instance, do left-wing groups discuss "fascism" more frequently than right-wing groups?
  2. Framing: This captures the moral context of a word. Drawing on Moral Foundations Theory (e.g., care/harm, sanctity/degradation), we project BERT embeddings into "moral subspaces" to determine if different groups associate the same word with different moral values.

The "secret sauce" of our approach is Structured Sparsity (specifically group lasso regularization). During training, the model is forced to ignore most words and only use a tiny subset (e.g., 150 out of 1,000) to predict the network structure. The words it chooses to keep are, by definition, the most polarized ones because they are the best predictors of which subreddits are ideologically aligned.

Datasets

  • Reddit Politosphere: A dataset of 605 political subreddits spanning 2008 to 2019. The network is built using "statistical backboning" on user overlaps—if two subreddits share a disproportionate number of users, they are linked.
  • DW-NOMINATE: For extrinsic evaluation, we used these scores (which measure the ideological leaning of US Congress members) to represent the "ground truth" political leaning of US states, then compared this to data from state-level subreddits (e.g., /r/TexasPolitics).

Results and Discussion

Our empirical evaluation demonstrates that the network itself can serve as a label. By treating link prediction as the objective, the model discovers polarization as a latent variable.

Performance and Efficiency

The full model (SF-SGAE) outperformed versions that used only salience or only framing, achieving a link prediction AUC of approximately 0.91. Crucially, the model maintained high accuracy even when restricted to a small number of concepts.

Link prediction performance (AUC) as a function of the number of selected concepts.

As shown in the figure above, the model proves that a small number of "hot button" issues drive most of the observable network polarization. This efficiency allows for the identification of key ideological markers like "fascist" and "mainstream."

Tracking Radicalization and Real-World Correlation

By analyzing the learned subreddit embeddings over time, we could automatically detect subreddits that shifted from moderate to extreme positions (e.g., /r/FreeSpeech moving toward a more right-wing orientation). Furthermore, the polarized concepts found by the model were significantly better at predicting the actual ideological leaning of US states (based on congressional voting records) than random or frequency-matched concepts.

Limitations

We acknowledge several constraints:

  • Inherent Bias: The model relies on BERT embeddings, which contain historical and social biases.
  • Demographics: The findings are specific to Reddit, which skews younger and more male.
  • Network Assumptions: While user overlap usually indicates homophily, it can sometimes indicate conflict (e.g., "brigading"), which might complicate the link prediction signal.

Summary updated

Abstract

The increasing polarization of online political discourse calls for computational tools that are able to automatically detect and monitor ideological divides in social media. Here, we introduce a minimally supervised method that directly leverages the network structure of online discussion forums, specifically Reddit, to detect polarized concepts. We model polarization along the dimensions of agenda setting and framing, drawing upon insights from moral psychology. The architecture we propose combines graph neural networks with structured sparsity and results in representations for concepts and subreddits that capture phenomena such as ideological radicalization and subreddit hijacking. We also create a new dataset of political discourse covering 12 years and more than 600 online groups with different ideologies.
BibTeX
@inproceedings{hofmann2022modeling,
  author = {Valentin Hofmann and Janet B. Pierrehumbert and Hinrich Schütze},
  title = {{Modeling Ideological Agenda Setting and Framing in Polarized Online Groups with Graph Neural Networks and Structured Sparsity}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
  doi = {10.18653/v1/2022.findings-naacl.41},
}
Figure