A Weakly Supervised Propagation Model for Rumor Verification and Stance Detection with Multiple Instance Learning

Ruichao Yang, Jing Ma, Hongzhan Lin, Wei Gao

Abstract

This paper presents a novel weakly supervised framework designed to jointly address rumor verification and stance detection on social media. By leveraging Multiple Instance Learning (MIL) and the structural information inherent in propagation trees, the proposed model learns to identify post-level stances using only claim-level veracity labels. This approach effectively bypasses the need for costly post-level annotations while achieving state-of-the-art performance on both tasks across several benchmark datasets.

Introduction

The rapid spread of rumors on social media platforms poses significant challenges to information integrity. Combating this requires two primary tasks: rumor verification (classifying a claim as true, false, or unverified) and stance detection (determining if a post supports, denies, queries, or comments on a claim). While these tasks are complementary—for instance, false rumors often trigger a high volume of "deny" stances—they are typically modeled independently.

Examples of rumor claims and post stances organized into 'bags' for Multiple Instance Learning.

The primary bottleneck in stance detection is the requirement for massive amounts of post-level labels. This research addresses this by asking if stance can be inferred using only the final verdict of a rumor. By treating the entire rumor conversation as a "bag" and individual posts as "instances," the authors apply MIL to learn these nuanced relationships without explicit post-level supervision.

Methodology

The core of the proposed approach is a tree-structured MIL framework that captures the evolution of opinions within a conversation. The model utilizes two primary components:

Tree-Based Representation and Attention

The conversation is modeled using Top-down (information flow from source to replies) and Bottom-up (aggregation from replies to source) propagation trees. This structure is vital for capturing "double negatives" and other contextual nuances where a post's stance depends on the post it is replying to. A hierarchical attention mechanism is employed to identify "vital" posts that contribute most to the rumor's veracity.

The Top-down Tree Attention framework for weakly supervised rumor and stance classification.

Binary Decomposition

To simplify the complex multi-class mapping, the problem is decomposed into multiple binary MIL classifiers. Each classifier focuses on a specific veracity-stance pair (e.g., "False Rumor" and "Deny"). The outputs are then aggregated via a discriminative attention layer to produce final multi-class predictions for both the claim and the individual posts.

Experimental Evaluation

The model was trained under weak supervision using the Twitter15, Twitter16, and PHEME datasets, which provide claim-level veracity but lack post-level stance labels. Evaluation was conducted on RumorEval2019-S and SemEval8, which contain the necessary annotations to verify performance on both tasks.

Results and Discussion

The proposed MIL models (TD-MIL and BU-MIL) demonstrated significant improvements over existing methods:

  • Stance Detection: Despite the lack of stance labels during training, the model achieved results comparable to fully supervised systems. On RumourEval2019-S, the Top-down model (TD-MIL) trained on PHEME achieved a Macro-F1 of 0.434.
  • Rumor Verification: The framework set new state-of-the-art benchmarks, with the TD-MIL model achieving an AUC of 0.917 on RumorEval2019-S, outperforming standard recursive neural networks (RvNN).

Key insights revealed that the Top-down approach generally superior performance because it considers the global path of a conversation rather than local subtree aggregations. Furthermore, the results confirm that veracity serves as an effective guide for stance detection; if a rumor is known to be false, the model can successfully back-propagate that knowledge to identify what a "denial" looks like in context.

Conclusion and Future Work

This research demonstrates that joint rumor and stance detection can be effectively performed with minimal supervision by leveraging the structural synergy of social media conversations. While performance is currently tied to the size of the training data (with the larger PHEME dataset yielding better results), future work will explore replacing standard GRU encoders with pre-trained language models like BERT and extending the model to more complex graph structures beyond simple trees.

Summary updated

Abstract

The diffusion of rumors on social media generally follows a propagation tree structure, which provides valuable clues on how an original message is transmitted and responded by users over time. Recent studies reveal that rumor verification and stance detection are two relevant tasks that can jointly enhance each other despite their differences. For example, rumors can be debunked by cross-checking the stances conveyed by their relevant posts, and stances are also conditioned on the nature of the rumor. However, stance detection typically requires a large training set of labeled stances at post level, which are rare and costly to annotate. Enlightened by Multiple Instance Learning (MIL) scheme, we propose a novel weakly supervised joint learning framework for rumor verification and stance detection which only requires bag-level class labels concerning the rumor's veracity. Specifically, based on the propagation trees of source posts, we convert the two multi-class problems into multiple MIL-based binary classification problems where each binary model is focused on differentiating a target class (of rumor or stance) from the remaining classes. Then, we propose a hierarchical attention mechanism to aggregate the binary predictions, including (1) a bottom-up/top-down tree attention layer to aggregate binary stances into binary veracity; and (2) a discriminative attention layer to aggregate the binary class into finer-grained classes. Extensive experiments conducted on three Twitter-based datasets demonstrate promising performance of our model on both claim-level rumor detection and post-level stance classification compared with state-of-the-art methods.
BibTeX
@inproceedings{yang2022weakly,
  author = {Ruichao Yang and Jing Ma and Hongzhan Lin and Wei Gao},
  title = {{A Weakly Supervised Propagation Model for Rumor Verification and Stance Detection with Multiple Instance Learning}},
  booktitle = {International ACM SIGIR Conference on Research and Development in Information Retrieval},
  year = {2022},
  doi = {10.1145/3477495.3531930},
}
Figure