Investigating Controversy Framing across Topics on Social Media

Maxwell Weinzierl, Sanda M. HarabagiuEMNLP 2025

TL;DR

This paper introduces a novel method, CWDA-FoC, for automatically discovering and articulating "Frames of Communication" (FoCs) that explain controversial topics on social media. This method allows for cross-topic analysis, enabling us to understand how controversy is framed on new topics by leveraging knowledge from existing, annotated datasets.

What Problem Does This Paper Address?

Social media has become a primary source of news and opinion, making it crucial to understand how controversial issues are framed. While previous work focused on identifying what aspects of a topic are discussed (frame aspects), this paper tackles a more complex challenge: understanding how these aspects are interpreted and explained. This involves not just identifying controversial problems (CPs) but also generating the "Frames of Communication" (FoCs) that provide causal interpretations for these problems. The authors aim to automate this process, which is currently labor-intensive and typically limited to a single topic.

Methods

The core contribution is the Corpus-Wide Discovery & Articulation of FoCs (CWDA-FoC) method. This approach is "top-down," processing an entire corpus of social media posts at once. It leverages recent advances in Large Language Models (LLMs) with long context windows and structured prompting. The key innovation is the Corpus-Wide One-Shot (CWOS) prompting method. This method uses a single, comprehensive demonstration derived from an existing "Reference Corpus" (annotated with FoCs and CPs for a known topic) to prompt an LLM to discover and articulate FoCs for a new, unseen topic. This allows for cross-domain transfer of framing knowledge.

Datasets

The paper utilizes several datasets:

  • Reference Datasets:
    • C19: Annotated with FoCs and CPs for COVID-19 vaccination hesitancy.
  • IMM: Annotated with CPs for immigration discourse, with newly annotated FoCs by communication experts.
  • Test Datasets:
    • AB (Abortion), CC (Climate Change), FM (Feminism): These datasets consist of social media posts from the SemEval-2016 Task 6, which are annotated with controversial problems but not with FoCs. They are used solely for evaluating the CWDA-FoC method.

Key Insights and Ideas

  • Cross-Topic Framing is Feasible: The CWDA-FoC method demonstrates that framing knowledge can be transferred across different controversial topics, a significant step towards understanding broader discourse patterns.
  • LLMs with Long Context are Game-Changers: The ability of newer LLMs to process massive context windows (100,000+ tokens) is essential for the CWDA-FoC method, allowing the entire reference corpus and test posts to be processed simultaneously.
  • One-Shot Demonstration Power: A single, well-crafted demonstration derived from a related topic can effectively guide LLMs to discover and articulate FoCs for entirely new topics, even without prior topic-specific annotations.
  • Reasoning-Focused LLMs Excel: Models specifically trained for reasoning (like o1 and o3-mini) showed superior performance, suggesting that the task benefits from the LLM's ability to "think" through the provided demonstration.
  • Quality of Reference Data Matters: The performance on test datasets was influenced by the quality and size of the reference corpus used for the one-shot demonstration. Larger and more comprehensive reference data led to better results.

Key Results

The CWDA-FoC method demonstrated strong performance across multiple topics (COVID-19 vaccines, immigration, abortion, climate change, feminism).

  • Soundness and Clarity: The method achieved high scores for the soundness (Z) and clarity (A) of the discovered FoCs, indicating that the generated frames accurately address the interpreted problems and are well-articulated.
  • Novelty: The method was able to uncover new frames (PA metric) that were not present in the reference datasets, demonstrating its ability to generalize and discover emergent framing patterns.
  • Comparison: While a prior bottom-up method (CoT-ICACL) performed well on its specific topic, CWDA-FoC showed comparable or superior results on new topics and offered greater scalability and flexibility.
  • Qualitative Examples: The paper provides concrete examples of discovered FoCs for abortion, climate change, and feminism, illustrating how the method captures nuanced interpretations of controversial problems. For instance, for abortion, a discovered FoC was: "Fetuses do not consent to death, and therefore abortion is murder."

Limitations and Future Work

  • Platform Specificity: The current method is tailored for social media content from Twitter/X. Its effectiveness on other platforms (e.g., Reddit) with different text lengths and structures needs further investigation.
  • Multimodality: The current approach relies solely on textual content. Future work aims to incorporate images, videos, and other multimedia elements to enable a more comprehensive frame discovery.
  • LLM Context Window Dependency: The CWOS prompting approach requires LLMs with very large context windows, which limits the use of some older or smaller models.
  • Broader Applicability: The authors intend to adapt the approach for broader applicability across diverse social media platforms and to explore its use in other domains beyond controversial topics.

Summary updated

BibTeX
@inproceedings{weinzierl2025investigating,
  author = {Maxwell Weinzierl and Sanda M. Harabagiu},
  title = {{Investigating Controversy Framing across Topics on Social Media}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure