GunStance: Stance Detection for Gun Control and Gun Regulation

Nikesh Gyawali, Iustin Sirbu, Tiberiu Sosea, Sarthak Khanal, Doina Caragea, Traian Rebedea, Cornelia Caragea

Introduction

Stance detection—the task of determining whether an author is "for," "against," or "neutral" toward a specific target—is a cornerstone of modern social media analysis. However, most research treats broad topics like "gun control" as a monolith. A new study introduces GunStance, a project that dives deeper into the linguistic and moral complexities of this debate following mass shooting events.

The Challenge of Nuance in Policy Debates

The gun control debate in the U.S. is notoriously polarized. Existing datasets often use generic targets like "guns," failing to capture the distinction between a "strong" stance (e.g., banning guns) and a "softer" stance (e.g., regulating guns). The researchers identified two primary challenges:

  1. Data Scarcity: Manually labeling thousands of tweets is expensive and slow.
  2. Linguistic Nuance: Stances are often expressed implicitly (e.g., "Thoughts and prayers" vs. "Policy and change"), making it difficult for standard models to generalize across different events.

Methodology: AUM-ST + ChatGPT

To address these challenges, the researchers developed a novel hybrid training approach using Area Under the Margin Self-Training (AUM-ST) combined with ChatGPT.

  • AUM-ST: A semi-supervised technique where a "teacher" model trains on labeled data and predicts labels for unlabeled data. It uses a "Margin" metric to gauge model confidence.
  • The Hybrid Twist: To prevent "pseudo-label" poisoning from uncertain "hard" examples, the system identifies ambiguous tweets (AUM score near zero) and consults ChatGPT for labeling.
  • Efficiency: By using the LLM for only the hardest 5% of cases, the researchers leveraged high-level reasoning without the prohibitive costs of full-dataset API labeling.

The GunStance Dataset

The GunStance dataset is the first large-scale collection specifically designed for stance detection regarding gun control in the wake of mass shootings.

  • Source: Tweets from seven major U.S. mass shooting events (2021–2022), including Uvalde, Buffalo, and Boulder.
  • Scale: 5,500 manually annotated tweets and over 16,000 unlabeled tweets.
  • Targets: Specifically split between "banning" and "regulating" targets to facilitate nuanced policy analysis.

Key Insights: Morality and Language

The study utilized Moral Foundation Theory to understand the underlying drivers of each stance.

Moral foundations and sentiment analysis across stances.

As shown in Figure 2, those In-Favor of gun bans lean heavily on "Care" and "Authority" (protecting lives, following rules). Conversely, those Against lean on "Fairness" (the right to self-defense and the perceived unfairness of punishing law-abiding citizens). Sentiment analysis also revealed that "Against" stances tend to be slightly more positive in sentiment, often focusing on rights and protection, while "In-Favor" stances are more neutral or negative, reflecting the tragedy of the events.

Word clouds representing common terms in 'In-Favor' and 'Against' stances.

Linguistically, "In-Favor" stances are frequently implicit, referencing "thoughts and prayers" as insufficient or mentioning specific tragedies like "Sandy Hook." "Against" stances are often more explicit, citing "strict laws," "second amendment" rights, or "mental health" as the primary issue rather than firearm access.

Results and Performance

The hybrid AUM-ST + ChatGPT model significantly outperformed traditional methods:

  • Superior Accuracy: It achieved an F1-score of 66.50 for the "Ban" target, surpassing supervised BERTweet (57.89) and zero-shot ChatGPT (63.62).
  • Generalization: In "leave-one-event-out" tests, the hybrid model outperformed vanilla self-training by 9-10%, suggesting it learns the underlying logic of the debate rather than just event-specific keywords.
  • Cross-Target Success: The model maintained high performance when trained on "banning" and tested on "regulating," demonstrating its ability to handle nuanced policy shifts.

Conclusion

GunStance provides a robust framework for understanding complex social media debates. By combining semi-supervised learning with targeted LLM consultation, the researchers created a model that is both cost-effective and highly capable of navigating the moral and linguistic nuances of the American gun debate.

Paper Citation: Gyawali, N., Sirbu, I., Sosea, T., Khanal, S., Caragea, D., Rebedea, T., & Caragea, C. (2024). GunStance: Stance Detection for Gun Control and Gun Regulation. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics.

Summary updated

Abstract

The debate surrounding gun control and gun regulation in the United States has intensified in the wake of numerous mass shooting events. As perspectives on this matter vary, it becomes increasingly important to comprehend individuals’ positions. Stance detection, the task of determining an author’s position towards a proposition or target, has gained attention for its potential use in understanding public perceptions towards controversial topics and identifying the best strategies to address public concerns. In this paper, we present GunStance, a dataset of tweets pertaining to shooting events, focusing specifically on the controversial topics of “banning guns” versus “regulating guns.” The tweets in the dataset are sourced from discussions on Twitter following various shooting incidents in the United States. Amazon Mechanical Turk was used to manually annotate a subset of the tweets relevant to the targets of interest (“banning guns” and “regulating guns”) into three classes: In-Favor, Against, and Neutral. The remaining unlabeled tweets are included in the dataset to facilitate studies on semi-supervised learning (SSL) approaches that can help address the scarcity of the labeled data in stance detection tasks. Furthermore, we propose a hybrid approach that combines curriculum-based SSL and Large Language Models (LLM), and show that the proposed approach outperforms supervised, semi-supervised, and LLM-based zero-shot models in most experiments on our assembled dataset.
BibTeX
@inproceedings{gyawali2024gunstance,
  author = {Nikesh Gyawali and Iustin Sirbu and Tiberiu Sosea and Sarthak Khanal and Doina Caragea and Traian Rebedea and Cornelia Caragea},
  title = {{GunStance: Stance Detection for Gun Control and Gun Regulation}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.acl-long.650},
}
Figure