Structure-aware Propagation Generation with Large Language Models for Fake News Detection

Mengyang Chen, Lingwei Wei, Wei Zhou, Songlin Hu

TL;DR

This paper introduces StruSP, a novel framework that uses Large Language Models (LLMs) to generate realistic synthetic propagation data for fake news detection. By explicitly modeling the structural patterns of how news spreads, StruSP significantly improves detection accuracy, especially in scenarios with limited real-world data.

What Problem Does This Paper Address?

The spread of fake news is a major societal challenge. While methods that analyze how news propagates (propagation-based methods) are effective, they often struggle when the available data is incomplete. Recent attempts to use LLMs to generate synthetic propagation data have shown promise, but they tend to produce propagation patterns that are too uniform and don't accurately reflect the complex, often irregular structures of real-world discussions. This mismatch limits their effectiveness in improving fake news detection. The paper aims to bridge this gap by creating synthetic propagation data that is both semantically plausible and structurally consistent with real-world propagation.

Methods

The core of the proposed solution is the StruSP framework, which has two main components:

  1. Bidirectional Evolutionary Propagation (BEP) Learning: This is the novel training strategy for LLMs.
  2. Structure-aware Hybrid Sampling: This module captures both the breadth-wise spread (e.g., sharing across many users) and depth-wise progression (e.g., layered discussions) of real propagation trees. It uses Breadth-First Search (BFS) and Depth-First Search (DFS) to sample propagation paths.
  3. Masked Propagation Modeling: Inspired by masked language modeling, this technique trains the LLM to reconstruct "masked" nodes within sampled propagation paths. By predicting preceding and succeeding nodes, the LLM learns to understand the structural dependencies and evolutionary patterns of propagation. This is done using a structured JSON format for input and output, making the LLM's interaction more robust.
  4. Structure-aware Propagation Enhancement (SPE): Once the LLM is trained with BEP, it's used to generate synthetic propagation data. This generated data is then used to enrich incomplete real propagation trees, creating a more comprehensive representation for the downstream fake news detection model.

Datasets

The paper evaluates its approach on three publicly available datasets:

  • Twitter: Contains tweets annotated as true or false.
  • CED: A Chinese rumor dataset scraped from Weibo, including forwarding and comment information.
  • PHEME5: Collections of rumors and non-rumors released on Twitter during specific events.

The datasets were split into training, validation, and testing sets in a 7:1:2 ratio.

Key Insights and Ideas

  • Structure Matters: The paper strongly emphasizes that simply generating semantically similar text isn't enough for fake news detection. The way information spreads (its structure) is crucial and often overlooked by previous LLM-based generation methods.
  • BEP for Robustness: The Bidirectional Evolutionary Propagation (BEP) learning strategy is key to making LLMs understand propagation dynamics. This deep structural learning makes the LLM less sensitive to prompt variations, a common issue in LLM applications.
  • Synthetic Data as a Powerful Augmentation: StruSP demonstrates that well-generated synthetic propagation data can significantly boost the performance of existing fake news detection models, especially in challenging scenarios like early detection or cross-platform generalization.
  • Bridging the Gap: The framework effectively bridges the gap between the often-uniform output of generic LLMs and the complex, irregular structures of real-world information diffusion.

Key Results

  • Improved Detection Performance: StruSP consistently improves fake news detection performance across various backbone models (GCN, Bi-GCN, EBGCN, RAGCL) on the tested datasets. For instance, StruSP w/RAGCL achieved state-of-the-art performance, with a 2.95% accuracy improvement over RAGCL on PHEME5.
  • Effective Early Detection: In early detection scenarios (where only limited propagation data is available), StruSP-enhanced models achieved performance close to using full propagation data, significantly outperforming methods that relied solely on limited real data or less structurally aware generation.
  • Cross-Platform Generalization: StruSP demonstrated strong performance in cross-platform detection, showing its ability to adapt to new platforms by leveraging learned structural patterns.
  • Realistic Propagation Generation: Qualitative analysis and quantitative metrics (Structural Entropy, Max Depth, Max Breadth, Semantic Consistency, Sentiment Consistency, Semantic Homogeneity) showed that StruSP-generated propagation patterns closely matched real data, outperforming baselines like DELL and GenFEND.

Limitations and Future Work

  • Data Dependency: The proposed StruSP framework relies on observed propagation data for training the LLM generator. This means its performance is sensitive to the quality and coverage of the training data.
  • Prompt Sensitivity: While BEP reduces sensitivity, prompt design still plays a role. Less expressive prompts might struggle to capture complex structural dependencies.
  • Lack of User-Level Modeling: The current framework doesn't explicitly model individual user behaviors or social dynamics, which could limit the realism and personalization of generated content, especially in emotionally charged conversations.
  • Future Work: The authors suggest exploring more advanced prompt engineering techniques and incorporating user-level behavior modeling to further enhance the realism and effectiveness of synthetic propagation generation. They also highlight the potential for misuse of synthetic generation for malicious purposes and emphasize the need for responsible research.

Summary updated

Abstract

The spread of fake news on social media poses a serious threat to public trust and societal stability. While propagation-based methods improve fake news detection by modeling how information spreads, they often suffer from incomplete propagation data. Recent work leverages large language models (LLMs) to generate synthetic propagation, but typically overlooks the structural patterns of real-world discussions. In this paper, we propose a novel structure-aware synthetic propagation enhanced detection (StruSP) framework to fully capture structural dynamics from real propagation. It enables LLMs to generate realistic and structurally consistent propagation for better detection. StruSP explicitly aligns synthetic propagation with real-world propagation in both semantic and structural dimensions. Besides, we also design a new bidirectional evolutionary propagation (BEP) learning strategy to better align LLMs with structural patterns of propagation in the real world via structure-aware hybrid sampling and masked propagation modeling objective. Experiments on three public datasets demonstrate that StruSP significantly improves fake news detection performance in various practical detection scenarios. Further analysis indicates that BEP enables the LLM to generate more realistic and diverse propagation semantically and structurally.
BibTeX
@inproceedings{chen2025structure,
  author = {Mengyang Chen and Lingwei Wei and Wei Zhou and Songlin Hu},
  title = {{Structure-aware Propagation Generation with Large Language Models for Fake News Detection}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure