A Joint Sentiment-Target-Stance Model for Stance Classification in Tweets

Javid Ebrahimi, Dejing Dou, Daniel LowdCOLING 2016 · OpenReview

Abstract

Classifying the stance expressed in online microblogging social media is an emerging problem in opinion mining. We propose a probabilistic approach to stance classification in tweets, which models stance, target of stance, and sentiment of tweet, jointly. Instead of simply conjoining the sentiment or target variables as extra variables to the feature space, we use a novel formulation to incorporate three-way interactions among sentiment-stance-input variables and three-way interactions among target-stance-input variables. The proposed specification intuitively aims to discriminate sentiment features from target features for stance classification. In addition, regularizing a single stance classifier, which handles all targets, acts as a soft weight-sharing among them. We demonstrate that discriminative training of this model achieves the state-of-the-art results in supervised stance classification, and its generative training obtains competitive results in the weakly supervised setting.

Introduction

Stance Classification (SC) is the task of determining if a writer is "in favor of," "against," or "neutral" toward a specific target. While it shares similarities with sentiment analysis, it is significantly more nuanced. For instance, a person might express a negative sentiment ("I hate the heat") while maintaining a positive stance toward a target ("...but I support climate change legislation").

The core challenges in SC include handling implicit targets (where the target is not explicitly mentioned), distinguishing between sentiment-bearing words and topic-defining words (feature overlap), and managing data scarcity, as manual annotation for stance is expensive.

Methodology: The STS Model

The authors propose the STS (Sentiment-Target-Stance) model, a log-linear approach that uses an undirected graphical model to capture dependencies between these variables.

Graphical representation of the STS model showing joint dependencies between sentiment (s), target (t), and stance (y) for input features (x_i).

Key innovations include:

  • Three-Way Interactions: The model uses two tensors to capture interactions between (Sentiment, Stance, Input) and (Target, Stance, Input). This allows the model to learn which features are sentiment-driven versus target-driven.
  • Soft Weight-Sharing: Instead of training separate classifiers for every target, a single classifier is used for all targets. Regularization effectively "shares" knowledge across targets, identifying generalized features of stance (e.g., what "favor" looks like across different domains).
  • Dual Training Modes: The model can be trained discriminatively (Disc-STS) for supervised tasks with clean labels, or generatively (Gen-STS) using Contrastive Divergence for weakly supervised tasks where labels are noisy or sparse.

Experimental Setup

The model was evaluated using the SemEval 2016 Task 6 datasets:

  • Task 6.A (Supervised): 4,163 tweets labeled for stance toward five targets: Atheism, Climate Change, Feminist Movement, Hillary Clinton, and Legalization of Abortion.
  • Task 6.B (Weakly Supervised): Focused on "Donald Trump," using a large unlabeled set (~78,000 tweets) and a small, manually labeled test set. A noisy training set was created by bootstrapping from stance-indicative hashtags.

Results and Discussion

In the supervised setting (Task 6.A), the discriminative version of the model (Disc-STS) achieved state-of-the-art performance, outperforming standard deep learning architectures.

Performance comparison on the SemEval 2016 Task 6.A supervised dataset.

As shown in the table above, Disc-STS achieved a 71.03 Micro F1 score, surpassing CNNs (67.33) and RNNs (67.82). The model's ability to discriminate between topic-specific n-grams (e.g., "campaign") and sentiment-bearing n-grams (e.g., "murder") proved crucial.

In the weakly supervised "Donald Trump" task (Task 6.B), the generative version (Gen-STS) demonstrated superior robustness, achieving an F-average of 56.73, compared to 47.51 for Disc-STS, which tended to overfit to noisy hashtag labels. Furthermore, the model successfully learned semantic relationships between targets, such as the similarity between "Feminism" and "Abortion."

Conclusion

The STS model provides a robust framework for stance classification by jointly modeling sentiment and target interactions. While the current approach relies on n-gram features and hashtag bootstrapping, future work could integrate these multi-way interaction logics into deep neural networks or incorporate user metadata to further enhance performance.

Summary updated

Abstract

Classifying the stance expressed in online microblogging social media is an emerging problem in opinion mining. We propose a probabilistic approach to stance classification in tweets, which models stance, target of stance, and sentiment of tweet, jointly. Instead of simply conjoining the sentiment or target variables as extra variables to the feature space, we use a novel formulation to incorporate three-way interactions among sentiment-stance-input variables and three-way interactions among target-stance-input variables. The proposed specification intuitively aims to discriminate sentiment features from target features for stance classification. In addition, regularizing a single stance classifier, which handles all targets, acts as a soft weight-sharing among them. We demonstrate that discriminative training of this model achieves the state-of-the-art results in supervised stance classification, and its generative training obtains competitive results in the weakly supervised setting.
BibTeX
@inproceedings{ebrahimi2016joint,
  author = {Javid Ebrahimi and Dejing Dou and Daniel Lowd},
  title = {{A Joint Sentiment-Target-Stance Model for Stance Classification in Tweets}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2016},
  url = {https://openreview.net/forum?id=QC39u0qWxG},
}
Figure