TL;DR
This paper introduces TASTE, a novel multimodal architecture for stance detection that effectively combines textual content with conversational structure. By fusing these two modalities using a Gated Residual Network (GRN), TASTE achieves state-of-the-art results, demonstrating the critical importance of understanding the broader conversational context for accurate stance identification.
What Problem Does This Paper Address?
Stance detection, the task of identifying a speaker's attitude towards a target, is crucial for various downstream applications like understanding the spread of misinformation and analyzing public opinion. While many existing methods focus solely on the textual content of an utterance, this paper argues that the conversational context is equally, if not more, important. Opinions are not formed in a vacuum; they evolve within a dynamic interaction between participants. Previous work has explored using conversational structure, but TASTE aims to create a more robust system by harmoniously integrating both text and structure.
Methods
TASTE is a multimodal architecture that leverages two key types of information:
- Textual Embedding: Utterances are represented using Sentence-BERT (S-BERT), a pre-trained transformer model known for its effectiveness in capturing sentence semantics.
- Structural Embedding: The conversational structure is transformed into an "interactions graph" where nodes represent speakers and edges represent interactions. The weight of these edges is determined by the type and frequency of interactions (replies, quotes). To derive speaker embeddings, TASTE employs a max-cut objective solved via a Semi-Definite Program (SDP). This approach aims to maximize the distance between speakers who are likely to have opposing stances.
These two embeddings are then fused using a Gated Residual Network (GRN). GRNs are particularly adept at combining information from different sources, dynamically weighting their contributions. This fusion is then fed into a simple MLP classifier for the final stance prediction. The authors also explore a simpler concatenation-based fusion as a baseline.
Datasets
The paper evaluates TASTE on two widely used datasets for stance detection:
- 4Forums: A collection of debates from online forums covering topics like abortion, evolution, gay marriage, and gun control. Stance labels are provided at the user level and broadcast to individual posts.
- CreateDebate: An online platform for debates where users self-annotate their stance. The dataset includes debates on topics such as abortion, gay rights, marijuana, and Obama.
Key Insights and Ideas
- Structure is King (Often): Surprisingly, the authors find that in many cases, using the conversational structure alone (via SDP) outperforms text-only models. This suggests that the dynamics of interaction can be a very strong signal for stance.
- Synergy of Text and Structure: While structure is powerful, combining it with textual content via TASTE provides a significant boost, adding an average of 12% accuracy. This highlights the complementary nature of the two modalities.
- GRN's Advantage: The Gated Residual Network (GRN) for fusion significantly outperforms simple concatenation. This indicates that a more nuanced approach to combining embeddings is crucial for effectively leveraging the distinct properties of text and structure.
- SDP's Superiority: The SDP-based structural embedding method proves more effective than node2vec for this task. The authors attribute this to SDP's design, which is geared towards separating opposing stances, aligning with the goal of stance detection.
Key Results
TASTE, specifically the TASTE@GRN(SDP+SBERT) variant, achieves state-of-the-art results on both the 4Forums and CreateDebate datasets across multiple topics. For instance, on the 4Forums dataset (author stance), TASTE@GRN(SDP+SBERT) reached an average accuracy of 81.2%, outperforming strong baselines like STEM (75.6%) and S-BERT (76.3%). On CreateDebate (author stance), it achieved 77.0%, also leading other models. The paper also shows that the GRN fusion strategy consistently outperforms concatenation.
Limitations and Future Work
- Dataset Diversity: The current evaluation is limited to two specific datasets. The authors suggest extending evaluation to more diverse and contemporary platforms like X (Twitter) or Reddit to improve generalizability.
- Language Dependence: The model's performance is likely tied to English conversational norms. Future work could explore cross-lingual adaptations.
- Stable Stance Assumption: The author-level stance detection assumes a stable stance throughout a discussion. This might not hold for platforms where users are encouraged to change their minds. Future work could address this by incorporating local turn-taking dynamics.
- Fusion Strategy Exploration: While GRN is shown to be effective, further exploration of fusion strategies could yield additional improvements.
Summary updated