Structure-aware Propagation Generation with Large Language Models for Fake News Detection

Mengyang Chen, Lingwei Wei, Wei Zhou, Songlin Hu

TL;DR

This paper introduces StruSP, a novel framework that uses Large Language Models (LLMs) to generate realistic synthetic propagation trees for fake news detection. By explicitly modeling the structural patterns of real-world news spread, StruSP significantly improves detection performance, especially in scenarios with limited real propagation data.

What Problem Does This Paper Address?

The spread of fake news is a major societal concern. While propagation-based methods have shown promise in detecting fake news by analyzing how information spreads, they often struggle with incomplete propagation data. Recent work has leveraged LLMs to generate synthetic propagation data to alleviate this scarcity. However, these LLM-based methods typically focus only on the semantic content of messages, neglecting the crucial structural patterns (like branching and depth) that characterize real-world information diffusion. This mismatch between generated and real propagation limits their effectiveness, particularly in early detection or cross-platform scenarios.

Methods

The core of StruSP lies in its Structure-aware Synthetic Propagation enhanced detection (StruSP) framework, which has two main components:

  1. Bidirectional Evolutionary Propagation (BEP) Learning: This is the key innovation. Instead of just generating text, BEP trains LLMs to understand and replicate the structure of news propagation.
  2. Structure-aware Hybrid Sampling: This module captures propagation dynamics by traversing real propagation trees using both Breadth-First Search (BFS) for wide dissemination and Depth-First Search (DFS) for layered discussions. This creates "propagation paths" that represent both breadth and depth.
  3. Masked Propagation Modeling: Inspired by masked language modeling, this technique trains the LLM to reconstruct masked nodes within these sampled propagation paths. By predicting preceding and succeeding nodes, the LLM learns to capture the structural dependencies and evolutionary patterns of real-world propagation.
  4. Structure-aware Propagation Enhancement (SPE): Once the LLM is trained with BEP, it's used to generate structurally coherent extensions for incomplete propagation trees. This synthetic data is then integrated with the original (partial) propagation data to enrich it for downstream fake news detection.

Datasets

The authors evaluated StruSP on three public datasets:

  • Twitter: Contains tweets annotated as true or false.
  • CED: Chinese rumor data scraped from Weibo, including forwarding and comment information.
  • PHEME5: Collections of rumors and non-rumors released on Twitter during specific events.

The datasets were split into training, validation, and testing sets in a 7:1:2 ratio.

Key Insights and Ideas

  • Structure Matters: The paper strongly emphasizes that simply generating semantically plausible text isn't enough for effective fake news detection. The structure of how information spreads is equally, if not more, important.
  • LLMs Can Learn Structure: The BEP strategy demonstrates that LLMs can be fine-tuned to understand and generate complex, realistic propagation structures, moving beyond their typical text-generation capabilities.
  • Synthetic Data Can Bridge Gaps: StruSP effectively uses LLM-generated propagation to augment limited real-world data, significantly boosting performance in challenging scenarios like early detection.
  • Robustness to Prompts: The BEP learning strategy makes the LLM's structural understanding robust to variations in prompt phrasing, meaning less effort is needed for prompt engineering.

Key Results

  • Improved Detection Performance: StruSP consistently improved the performance of various baseline fake news detection models (GCN, Bi-GCN, EBGCN, RAGCL) across all three datasets. For example, StruSP w/RAGCL achieved state-of-the-art performance, gaining 2.95% accuracy improvement over RAGCL on PHEME5.
  • Effective Early Detection: In early detection scenarios with very limited propagation data, StruSP-enhanced models achieved performance close to using the full propagation data, demonstrating its ability to generate informative and structurally consistent synthetic data.
  • Cross-Platform Generalization: StruSP enabled better fake news detection across different platforms (e.g., training on Twitter, testing on Weibo), highlighting the importance of structure-aware generation for domain adaptation.
  • Realistic Propagation Generation: Both macro-level and micro-level analyses showed that StruSP-generated propagation trees were more structurally and semantically aligned with real-world propagation compared to other LLM-based generation methods like DELL and GenFEND.

Limitations and Future Work

  • Data Dependency: StruSP relies on observed propagation data for training the LLM generator, making its performance sensitive to the quality and coverage of this training data.
  • Prompt Sensitivity (Mitigated): While the BEP strategy reduces sensitivity, prompt design still plays a role. Less expressive prompts might struggle with complex structures.
  • Lack of User-Level Modeling: The current framework doesn't explicitly model individual user behaviors or social dynamics, which could limit the realism and personalization of generated content, especially in emotionally charged conversations.
  • Future Directions: The authors suggest exploring more sophisticated ways to model user-level behaviors and social dynamics to further enhance the realism of generated propagation. They also highlight the need for continued research into robust LLM evaluation and responsible use of synthetic data generation.

Summary updated

Abstract

The spread of fake news on social media poses a serious threat to public trust and societal stability. While propagation-based methods improve fake news detection by modeling how information spreads, they often suffer from incomplete propagation data. Recent work leverages large language models (LLMs) to generate synthetic propagation, but typically overlooks the structural patterns of real-world discussions. In this paper, we propose a novel structure-aware synthetic propagation enhanced detection (StruSP) framework to fully capture structural dynamics from real propagation. It enables LLMs to generate realistic and structurally consistent propagation for better detection. StruSP explicitly aligns synthetic propagation with real-world propagation in both semantic and structural dimensions. Besides, we also design a new bidirectional evolutionary propagation (BEP) learning strategy to better align LLMs with structural patterns of propagation in the real world via structure-aware hybrid sampling and masked propagation modeling objective. Experiments on three public datasets demonstrate that StruSP significantly improves fake news detection performance in various practical detection scenarios. Further analysis indicates that BEP enables the LLM to generate more realistic and diverse propagation semantically and structurally.
BibTeX
@inproceedings{chen2025structure,
  author = {Mengyang Chen and Lingwei Wei and Wei Zhou and Songlin Hu},
  title = {{Structure-aware Propagation Generation with Large Language Models for Fake News Detection}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure