LLM-Driven Implicit Target Augmentation and Fine-Grained Contextual Modeling for Zero-Shot and Few-Shot Stance Detection

Yanxu Ji, Jinzhong Ning, Yijia Zhang, Zhi Liu, Hongfei LinEMNLP 2025

TL;DR

This paper introduces a novel framework for stance detection that tackles the challenge of implicit targets and the need for dynamic context-aware modeling. By leveraging Large Language Models (LLMs) to augment training data with previously overlooked implicit targets and employing a dynamic attention mechanism, the authors achieve state-of-the-art results in zero-shot and few-shot stance detection.

What Problem Does This Paper Address?

Stance detection aims to understand the attitude expressed in text towards a specific target. While recent work has focused on zero-shot and few-shot scenarios (predicting stances for unseen targets), two key limitations persist. Firstly, existing datasets often prioritize explicitly mentioned targets, neglecting implicitly inferred but semantically crucial targets (e.g., "democracy" in a text about corporate regulation). This omission hinders models' ability to grasp nuanced meanings, especially in cases of metaphor or irony. Secondly, current models often use fixed interactions between text and target representations, failing to adaptively adjust their relative importance based on context. This inflexibility limits their generalization capabilities.

Methods

The authors propose a two-stage framework:

  1. Hierarchical Collaborative Target Augmentation (HCTA): This LLM-driven approach addresses the under-utilization of implicit targets.
  2. Implicit Target Extraction (ITE): It uses Chain-of-Thought (CoT) prompting with LLMs to identify semantically relevant but unannotated implicit targets.
  3. Stance Prediction via LLM Collaboration: Multiple LLMs then predict stances for these newly identified targets. A majority voting strategy is employed to aggregate these predictions, mitigating individual LLM biases and ensuring prediction reliability. This process generates an augmented dataset with pseudo-labels for implicit targets.
  4. Dynamic Multi-level Context-aware Attention Network (DyMCA): This network enhances contextual adaptability for text-target interactions.
  5. Joint Text-Target Representation (JTTR): A BART encoder creates a unified representation of text and target.
  6. Content-aware Mechanism (CM): This module dynamically adjusts the contributions of text and target based on their contextual relevance. It calculates a "w" score (between 0 and 1) representing the text's relative contribution, which is then used to adaptively weight the fused text-target representation. A residual target term is included to provide a stable semantic anchor.
  7. Stance Predictor: A classifier then predicts the stance probabilities.

Datasets

The primary dataset used for training and evaluation is the Varied Stance Topics (VAST) dataset. This dataset is noted for containing thousands of targets across diverse domains and is commonly used for ZSSD and FSSD tasks. The paper also mentions using a subset of the VAST dataset (10% of training data) for low-resource scenario evaluations.

Key Insights and Ideas

  • LLMs are crucial for implicit target discovery: The paper strongly demonstrates that LLMs, especially with CoT prompting, are significantly better at identifying implicit targets than non-LLM approaches.
  • Collaboration improves LLM reliability: Using multiple LLMs and a voting mechanism for stance prediction on implicit targets significantly reduces individual model biases and improves overall accuracy.
  • Dynamic context matters: The DyMCA architecture's ability to adaptively weigh text and target contributions based on context is key to its improved performance, especially in challenging scenarios.
  • Implicit targets are vital for generalization: Augmenting datasets with implicitly identified targets, as done by HCTA, leads to substantial performance gains, particularly in zero-shot and few-shot settings.

Key Results

The proposed DyMCA framework with HCTA achieves state-of-the-art performance on the VAST dataset for both Zero-Shot and Few-Shot Stance Detection.

  • Overall Macro F1 scores: 0.81 (Zero-Shot), 0.771 (Few-Shot), and 0.776 (ALL). These represent improvements of 1.12%, 3.63%, and 2.37% over the strongest competitor (TTS).
  • Low-resource scenario: With only 10% of training data, DyMCA (without HCTA) shows a 9.5% improvement in zero-shot performance over the best competitor. DyMCA with HCTA achieves a macro-F1 score of 0.801 in this setting.
  • Ablation studies: Removing HCTA leads to significant performance degradation, highlighting the importance of LLM-driven implicit target mining. Removing the Content-aware Mechanism (w/o CM) also causes performance drops, validating the adaptive weighting strategy.

Limitations and Future Work

  • LLM Reliability: The accuracy of implicit target extraction and pseudo-labeling is inherently tied to the reasoning quality and consistency of LLMs, which can vary.
  • Computational Cost: Using multiple LLMs for collaboration can be computationally expensive, posing challenges for large-scale applications.
  • Future Work: The authors suggest exploring more efficient and reliable target augmentation strategies, as well as principled methods for uncertainty-aware filtering of LLM outputs. They also propose investigating more efficient LLM collaboration techniques and exploring LLMs for more complex stance detection tasks.

Summary updated

BibTeX
@inproceedings{ji2025llm,
  author = {Yanxu Ji and Jinzhong Ning and Yijia Zhang and Zhi Liu and Hongfei Lin},
  title = {{LLM-Driven Implicit Target Augmentation and Fine-Grained Contextual Modeling for Zero-Shot and Few-Shot Stance Detection}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure