A veracity dissemination consistency-based few-shot fake news detection framework by synergizing adversarial and contrastive self-supervised learning

Weiqiang Jin, Ningwei Wang, Tao Tao, Bohang Shi, Haixia Bi, Biao Zhao, Hao Wu, Haibin Duan, Guang YangScientific 2024 · DOI

TL;DR

DetectYSF (Detection Yet See Few) is a novel few-shot fake news detection framework that thrives in data-scarce environments. By combining prompt-tuning with adversarial and contrastive self-supervised learning—and layering in a "social consistency" theory—it significantly outperforms state-of-the-art models when only a handful of labeled examples are available.

What Problem Does This Paper Address?

Fake News Detection (FEND) is a high-stakes arms race. While deep learning models have become quite good at it, they typically suffer from a "data hunger" problem, requiring massive, expertly annotated datasets to perform well. In the real world, news moves fast, and obtaining high-quality labels is slow, expensive, and labor-intensive.

When faced with "few-shot" scenarios (where only 16 to 128 labeled examples exist), traditional fine-tuning often fails to generalize. Furthermore, many models focus strictly on the text, ignoring the rich social context of how news is disseminated. The authors aim to bridge this gap by creating a model that learns robust representations from very little data while leveraging the social patterns of news sharing.

Methods

The authors introduce DetectYSF, which moves away from standard classification heads in favor of a more sophisticated architecture:

  1. Masked LM-based Prompt Tuning: Instead of traditional fine-tuning, the model treats news detection as a "fill-in-the-blank" task. It uses a template like: "It is [MASK] that [News Content]." The model then predicts if the mask should be "real" or "fake," better utilizing the pre-trained knowledge within models like BERT or RoBERTa.
  2. Sentence-Level Contrastive Learning: To make the model's internal representations more robust, the authors use contrastive learning. This forces the model to pull "positive" pairs (different views of the same news or news with the same label) closer together in vector space while pushing "negative" pairs (different news or different labels) further apart.
  3. Sample-Level Adversarial Learning (GAN): They integrate a Generative Adversarial Network (GAN) with two distinct generators. One generator uses random noise, while the other uses an independent PLM encoder to create "negative" fake news samples. This forces the primary detector to become more resilient against deceptive or noisy inputs.
  4. Veracity Feature Fusion: This is a post-processing step based on the authors' "news veracity dissemination consistency" theory. They build a graph of news-sharing behavior and use a sub-graph aggregation algorithm to refine a prediction based on the veracity of neighboring news nodes in the social network.

Datasets

The framework was evaluated on three widely recognized FEND benchmarks:

  • PolitiFact: High-quality political news data.
  • GossipCop: Entertainment and celebrity news.
  • FANG: A dataset focused on the social context and relationships between users, articles, and sources.

The experiments were conducted in 16, 32, 64, and 128-shot settings to simulate extreme data scarcity.

Key Insights and Ideas

  • The "Birds of a Feather" Theory: The authors formalize "news veracity dissemination consistency." They argue that social circles (echo chambers) tend to share news with a consistent level of truthfulness. If a user's network predominantly shares verified news, a new, unknown article shared by that network is statistically more likely to be genuine.
  • Prompting > Fine-tuning: For few-shot tasks, prompt-tuning is superior because it aligns the downstream task with the original pre-training objective (Masked Language Modeling) of the backbone Transformer.
  • Synergistic Learning: The paper demonstrates that contrastive learning (for better semantic clusters) and adversarial learning (for robustness) are complementary, not redundant.

Key Results

DetectYSF consistently outperformed all baselines, including specialized few-shot models like PET and KPT.

  • Massive Gains: On the GossipCop dataset, DetectYSF achieved 83.94% accuracy in a few-shot setting, a staggering improvement over PET (62.97%) and KPT (62.19%).
  • Social Context Matters: Ablation studies showed that the "Veracity Feature Fusion" (the social graph component) often provided the single largest boost in accuracy, proving that who shares the news is often as important as what the news says.
  • Robustness: The adversarial components allowed the model to maintain higher accuracy even when the input text was noisy or intentionally deceptive.

Limitations and Future Work

While powerful, the authors acknowledge several areas for growth:

  • Multimodality: The current version is text-centric. Future iterations should incorporate images and videos, which are central to modern misinformation.
  • Cross-Lingual Capabilities: The model needs to be tested in multilingual environments to address the global nature of fake news.
  • Real-time Scalability: The complex graph-based post-processing and adversarial training need to be optimized for real-time deployment on high-traffic social media platforms.

Summary updated

Abstract

With the rapid growth of social media, fake news (rumors) are rampant online, seriously endangering the health of mainstream social consciousness. Fake news detection (FEND), as a machine learning solution for automatically identifying fake news on Internet, is increasingly gaining the attentions of academic community and researchers. Recently, the mainstream FEND approaches relying on deep learning primarily involves fully supervised fine-tuning paradigms based on pre-trained language models (PLMs), relying on large annotated datasets. In many real scenarios, obtaining high-quality annotated corpora are time-consuming, expertise-required, labor-intensive, and expensive, which presents challenges in obtaining a competitive automatic rumor detection system. Therefore, developing and enhancing FEND towards data-scarce scenarios is becoming increasingly essential. In this work, inspired by the superiority of semi-/self- supervised learning, we propose a novel few-shot rumor detection framework based on semi-supervised adversarial learning and self-supervised contrastive learning, named Detection Yet See Few (DetectYSF). DetectYSF synergizes contrastive self-supervised learning and adversarial semi-supervised learning to achieve accurate and efficient FEND capabilities with limited supervised data. DetectYSF uses Transformer-based PLMs (e.g., BERT, RoBERTa) as its backbone and employs a Masked LM-based pseudo prompt learning paradigm for model tuning (prompt-tuning). Specifically, during DetectYSF training, the enhancement measures for DetectYSF are as follows: (1) We design a simple but efficient self-supervised contrastive learning strategy to optimize sentence-level semantic embedding representations obtained from PLMs; (2) We construct a Generation Adversarial Network (GAN), utilizing random noises and negative fake news samples as inputs, and employing Multi-Layer Perceptrons (MLPs) and an extra independent PLM encoder to generate abundant adversarial embeddings. Then, incorporated with the adversarial embeddings, we utilize semi-supervised adversarial learning to further optimize the output embeddings of DetectYSF during its prompt-tuning procedure. From the news veracity dissemination perspective, we found that the authenticity of the news shared by these collectives tends to remain consistent, either mostly genuine or predominantly fake, a theory we refer to as "news veracity dissemination consistency". By employing an adjacent sub-graph feature aggregation algorithm, we infuse the authenticity characteristics from neighboring news nodes of the constructed veracity dissemination network during DetectYSF inference. It integrates the external supervisory signals from "news veracity dissemination consistency" to further refine the news authenticity detection results of PLM prompt-tuning, thereby enhancing the accuracy of fake news detection. Furthermore, extensive baseline comparisons and ablated experiments on three widely-used benchmarks demonstrate the effectiveness and superiority of DetectYSF for few-shot fake new detection under low-resource scenarios.
BibTeX
@inproceedings{jin2024veracity,
  author = {Weiqiang Jin and Ningwei Wang and Tao Tao and Bohang Shi and Haixia Bi and Biao Zhao and Hao Wu and Haibin Duan and Guang Yang},
  title = {{A veracity dissemination consistency-based few-shot fake news detection framework by synergizing adversarial and contrastive self-supervised learning}},
  booktitle = {Scientific},
  year = {2024},
  doi = {10.1038/s41598-024-70039-9},
}
Figure