Transferring Structure Knowledge: A New Task to Fake News Detection towards Cold-Start Propagation

Lingwei Wei, Dou Hu, Wei Zhou, Songlin Hu

TL;DR

This paper introduces "cold-start fake news detection," a new challenge where models must detect fake news using only content when propagation data is unavailable. They propose a Structure Adversarial Net (SAN) framework to transfer knowledge learned from propagation trees to these content-only scenarios, significantly improving detection performance.

What Problem Does This Paper Address?

Traditional fake news detection methods often leverage both the content of a news article and its propagation structure (how it spreads across social networks) to achieve high accuracy. However, in real-world applications, propagation data is frequently missing, especially for newly emerging news or in scenarios where tracking spread is difficult. This "cold-start" problem means that models trained on rich propagation data perform poorly when faced with content-only samples. The paper addresses this gap by proposing a new task and a method to enable propagation-based models to generalize to situations without propagation information.

Methods

The core of the paper's contribution is the Structure Adversarial Net (SAN) framework. SAN is designed to be applied on top of existing propagation-based fake news detection models. The key idea is to learn features that are "structure-invariant," meaning they are useful for detection regardless of whether propagation data is present or not.

Here's how it works:

  1. Feature Extractor: An encoder learns representations from both content and propagation data.
  2. Fake News Classifier: This standard component uses the learned representations to classify news as fake or real.
  3. Structure Discriminator: This is the novel addition. It's trained to predict whether a given representation was derived from data with propagation or without it.
  4. Adversarial Training: The feature extractor is trained to fool the structure discriminator. This forces the extractor to produce representations that are similar for both propagation-present and propagation-absent data, effectively making the features transferable. A gradient reversal layer is used to achieve this adversarial training.

The overall objective is to minimize the classification loss (correctly identifying fake news) while simultaneously minimizing the dissimilarity loss captured by the structure discriminator.

Datasets

The authors evaluate their approach on three public datasets:

  • PolitiFact: Contains news articles from the fact-checking website PolitiFact.
  • GossipCop: Contains news articles from the gossip-focused website GossipCop.
  • PHEME-5: Contains tweets related to five different events, allowing for both general and event-specific cold-start scenarios.

For the "general cold-start" setup, they randomly split data into training and testing sets, removing propagation trees from the test set. For the "event-aware cold-start" setup (using PHEME-5), they use data from one event for testing and the rest for training, again removing propagation from the test set.

Key Insights and Ideas

  • Propagation Data is a Double-Edged Sword: While propagation structure significantly boosts fake news detection when available, it can also lead to models that are overly specialized and fail dramatically when that structure is missing.
  • The "Cold-Start" Challenge is Real: Existing propagation-based models suffer significant performance degradation when applied to content-only samples, highlighting the need for new approaches.
  • Structure-Invariant Features are Key: The core insight is that by forcing models to learn representations that are indistinguishable regardless of propagation data presence, we can achieve better generalization.
  • Adversarial Learning for Transferability: Using a discriminator to identify structure-specific features and then training the main model to fool it is an effective way to learn transferable representations.

Key Results

  • Significant Improvement: The SAN framework consistently improved the performance of various baseline propagation-based models across all tested datasets and task setups (general and event-aware cold-start). For instance, on PolitiFact in the general cold-start setting, UPSR+SAN achieved a +2.8% improvement in weighted-F1 score over UPSR.
  • Challenge Confirmed: The results clearly demonstrate the difficulty of the cold-start fake news detection task, with baseline models showing considerable performance drops compared to their general detection performance.
  • Qualitative Visualization: t-SNE visualizations showed inconsistent representation distributions for training and testing samples in the cold-start scenario, visually confirming the generalization issue that SAN aims to solve.

Limitations and Future Work

  • Model Complexity: The paper notes that more complex propagation-based models tend to suffer greater performance degradation in cold-start scenarios, suggesting that simpler, more robust feature extraction might be beneficial.
  • Future Directions: The authors suggest exploring further directions to promote the development of cold-start fake news detection, though specific details are not elaborated upon in the provided text. They also mention the need to investigate different settings for cold-start fake news detection.

Summary updated

Abstract

Many fake news detection studies have achieved promising performance by extracting effective semantic and structure features from both content and propagation trees. However, it is challenging to apply them to practical situations, especially when using the trained propagation-based models to detect news with no propagation data. Towards this scenario, we study a new task named cold-start fake news detection, which aims to detect content-only samples with missing propagation. To achieve the task, we design a simple but effective Structure Adversarial Net (SAN) framework to learn transferable features from available propagation to boost the detection of content-only samples. SAN introduces a structure discriminator to estimate dissimilarities among learned features with and without propagation, and further learns structure-invariant features to enhance the generalization of existing propagation-based methods for content-only samples. We conduct qualitative and quantitative experiments on three datasets. Results show the challenge of the new task and the effectiveness of our SAN framework.
BibTeX
@inproceedings{wei2024transferring,
  author = {Lingwei Wei and Dou Hu and Wei Zhou and Songlin Hu},
  title = {{Transferring Structure Knowledge: A New Task to Fake News Detection towards Cold-Start Propagation}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2024},
  url = {https://openreview.net/forum?id=ohQAcqwF1R},
}
Figure