Modeling Framing in Immigration Discourse on Social Media

Julia Mendelsohn, Ceren Budak, David Jurgens

Abstract

The framing of political issues can influence policy and public opinion. Even though the public plays a key role in creating and spreading frames, little is known about how ordinary people on social media frame political issues. By creating a new dataset of immigration-related tweets labeled for multiple framing typologies from political communication theory, we develop supervised models to detect frames. We demonstrate how users' ideology and region impact framing choices, and how a message's framing influences audience responses. We find that the more commonly-used issue-generic frames obscure important ideological and regional patterns that are only revealed by immigration-specific frames. Furthermore, frames oriented towards human interests, culture, and politics are associated with higher user engagement. This large-scale analysis of a complex social and linguistic phenomenon contributes to both NLP and social science research.

Introduction

In political communication, "framing" refers to the process of highlighting specific aspects of an issue to promote a particular interpretation or solution. While previous Natural Language Processing (NLP) research has focused on "issue-generic" frames—broad categories like Economic or Legality that apply to many topics—these often miss the nuance of specific debates. Furthermore, most framing research has focused on "elites," such as politicians and news media. This study addresses this gap by analyzing how the general public frames immigration on Twitter, how those choices vary by political leaning and geography, and how different frames resonate with audiences.

Methodology and Data

The researchers adopted a supervised learning approach to detect frames across three distinct typologies derived from political communication theory: Issue-Generic Policy frames, Immigration-Specific frames, and Narrative frames.

Table 1: Definitions and descriptions of the three framing typologies used in the study, including generic policy frames, immigration-specific frames, and narrative structures.

Datasets

  • Primary Corpus: 2.66 million English-language tweets (2018-2019) containing immigration-related keywords from users in the US, UK, and EU.
  • Annotated Dataset: A novel, manually labeled subset of 4,500 tweets. 3,600 were used for training, while 900 were consensus-coded by multiple trained annotators for evaluation.
  • Metadata: User ideology for US users was estimated using a Bayesian Spatial Following model, and location was inferred using existing geolocation tools.

Technical Approach

The team utilized a RoBERTa-base model, which was first "domain-fine-tuned" using masked-language modeling on the entire 2.6 million tweet corpus. The model was then trained on the annotated dataset for multi-label classification. Mixed-effects logistic and linear regression models were subsequently used to analyze the relationships between frames, user metadata, and engagement metrics.

Key Results and Insights

Model Performance

The fine-tuned RoBERTa model significantly outperformed baselines, achieving an overall F1 score of 0.657. Narrative and generic frames were easier to detect (F1 ~0.71-0.81) than the sparser, more nuanced immigration-specific frames (F1 ~0.55).

The "Masking" Effect of Generic Frames

A major finding of this research is that generic frames often obscure critical ideological nuances. For example, the generic frame Cultural Identity shows only a slight liberal bias. However, when examining specific frames, a massive divide appears: liberals frequently use Hero: Cultural Diversity, while conservatives favor Threat: National Cohesion. Relying solely on generic frames hides the most polarized aspects of the conversation.

Figure 1: Beta coefficients showing the relationship between specific frames and user ideology. Negative values indicate a liberal lean, while positive values indicate a conservative lean.

Ideological and Regional Patterns

  • Ideological Consistency: Models were more accurate at predicting frames in conservative tweets than liberal ones, suggesting that conservative framing on social media is more linguistically consistent and "hegemonic."
  • Regional Divergence: US-based tweets were significantly more likely to use "extreme" frames related to Crime, Security, and Fiscal Threats compared to the UK and EU, where Cultural Identity and Labor frames were more prevalent.

Alignment vs. Amplification

Different frames drive distinct types of engagement. Frames centered on human interest and morality (e.g., Victim: Discrimination) are associated with more favorites, suggesting personal alignment. In contrast, frames focusing on political strategy or perceived urgency (e.g., Public Order, Political Factors) drive more retweets, suggesting a desire to amplify a message for a broader cause.

Conclusion

This study highlights the necessity of using granular, issue-specific framing typologies to understand political discourse on social media. While generic frames provide a broad overview, they fail to capture the ideological and regional nuances that drive public debate. Future work should incorporate multi-modal data, such as images and video, and explore how framing evolves dynamically in response to real-world events.

Summary updated

Abstract

The framing of political issues can influence policy and public opinion. Even though the public plays a key role in creating and spreading frames, little is known about how ordinary people on social media frame political issues. By creating a new dataset of immigration-related tweets labeled for multiple framing typologies from political communication theory, we develop supervised models to detect frames. We demonstrate how users' ideology and region impact framing choices, and how a message's framing influences audience responses. We find that the more commonly-used issue-generic frames obscure important ideological and regional patterns that are only revealed by immigration-specific frames. Furthermore, frames oriented towards human interests, culture, and politics are associated with higher user engagement. This large-scale analysis of a complex social and linguistic phenomenon contributes to both NLP and social science research.
BibTeX
@inproceedings{mendelsohn2021modeling,
  author = {Julia Mendelsohn and Ceren Budak and David Jurgens},
  title = {{Modeling Framing in Immigration Discourse on Social Media}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
  url = {https://openreview.net/forum?id=4xgTCGQLQbS},
}
Figure