Abstract
Stance detection is an active task in natural language processing (NLP) that aims to identify the author's stance towards a particular target within a text. Given the remarkable language understanding capabilities and encyclopedic prior knowledge of large language models (LLMs), how to explore the potential of LLMs in stance detection has received significant attention. Unlike existing LLM-based approaches that focus solely on fine-tuning with large-scale datasets, we propose a new prompting method, called Chain of Stance (CoS). In particular, it positions LLMs as expert stance detectors by decomposing the stance detection process into a series of intermediate, stance-related assertions that culminate in the final judgment. This approach leads to significant improvements in classification performance. We conducted extensive experiments using four SOTA LLMs on the SemEval 2016 dataset, covering the zero-shot and few-shot learning setups. The results indicate that the proposed method achieves state-of-the-art results with an F1 score of 79.84 in the few-shot setting.
Introduction
Stance detection involves determining whether the author of a text is in favor of, against, or neutral toward a specific target (e.g., "Climate Change" or "Atheism"). While conceptually simple, it is notoriously difficult for automated systems because human language is frequently characterized by sarcasm, cultural nuances, and indirect expressions.
Current approaches generally rely on extensive fine-tuning or direct "black-box" prompting. Fine-tuning is computationally expensive, while direct prompting often fails to capture the underlying logic required for nuanced social media text. The authors address this gap by leveraging the structured reasoning capabilities of LLMs through a novel framework that makes the decision-making process both more accurate and more interpretable.
The Chain of Stance (CoS) Framework
The core innovation of this research is the Chain of Stance (CoS) framework. Inspired by "Chain of Thought" (CoT) prompting, CoS applies structured reasoning to social linguistics. Instead of requesting a final label immediately, the framework guides the LLM through six specific analytical steps:
- Contextual Understanding: Identifying the topic, author identity, and socio-cultural background.
- Main Idea Interpretation: Extracting core viewpoints and intentions.
- Emotional Analysis: Analyzing tone (e.g., sarcasm) and rhetorical devices.
- Stance Comparison: Explicitly comparing the text against the three possible labels.
- Logical Inference: Checking the consistency and rationality of the evidence.
- Final Decision: Synthesizing all previous steps into a final stance polarity.
Experimental Setup and Results
The researchers evaluated CoS using the SemEval-2016 Task 6 dataset, a standard benchmark containing 4,163 tweets across five targets: Hillary Clinton, the Feminist Movement, Legalization of Abortion, Atheism, and Climate Change. The models tested included Mistral-7B, Qwen1.5-7B, LLaMA 3-8B, and LLaMA 2-7B.
Experimental results demonstrate that the CoS method achieved state-of-the-art (SOTA) performance. In the Zero-Shot setting, it achieved a leading F1 score of 76.43 (using Mistral-7B). In the Few-Shot setting, the F1 score reached 79.84, surpassing existing specialized models like CoSD and MB-Cal.
Key Insights
- Structure Beats Scale: The Mistral-7B model using CoS outperformed much larger models and complex fine-tuned baselines, suggesting that the reasoning structure is as critical as parameter count.
- Emotion as a Driver: Explicitly forcing the model to analyze "emotional inclination" and "rhetorical devices" was the primary driver in improving accuracy for social media text, which is often sarcastic.
- Interpretability: Because the model outputs its reasoning for each of the six steps, researchers can pinpoint exactly where a model misclassified a tweet, such as understanding sentiment but failing to grasp cultural context.
Conclusion
Chain of Stance transforms LLMs into expert stance detectors by enforcing a sequential reasoning process that prioritizes human emotion and context. While the method increases computational overhead due to longer prompts, the gains in accuracy and interpretability represent a significant advancement in NLP stance detection. Future work will focus on optimizing prompt length and exploring domain-specific expert personas within the chain.
Summary updated