Contradicted in Reliable, Replicated in Unreliable: Dual-Source Reference for Fake News Early Detection

Yifan Feng, Weimin Li, Yue Wang, Jingchao Wang, Fangfang Liu, Zhongming HanAAAI 2025 · DOI

TL;DR

This paper introduces RUSR, a novel fake news detection method that leverages the contrasting semantic characteristics of news from reliable and unreliable sources. By analyzing the "support degree" from reliable sources and the "semantic outlier degree" within unreliable sources, RUSR achieves superior early detection of fake news, particularly in dynamic news environments.

What Problem Does This Paper Address?

The proliferation of fake news poses a significant threat to individuals and society. While existing fake news detection methods often rely on the support provided by reliable sources, they overlook a crucial aspect: the distinct ways fake news and real news manifest within the ecosystem of unreliable sources. Fake news, often crafted to promote specific agendas, tends to create a tightly clustered semantic space among unreliable sources that share its propaganda. Conversely, real news, even when discussed by unreliable sources, might exhibit more semantic diversity. This paper aims to bridge this gap by proposing a method that exploits this difference in semantic outlier degrees within unreliable sources, in addition to the traditional reliance on reliable sources. The challenge is to develop a robust detection system that can operate effectively in the early stages of news dissemination, where information is still evolving.

Methods

The core of the proposed method is the Reliable-Unreliable Source Reference (RUSR) framework. RUSR constructs a "publication background" for a given news item, consisting of related news with similar main objects of description and slightly earlier publication times, sourced from both reliable and unreliable outlets.

Key components of RUSR include:

  • Strongly Preference-Driven Support Degree Evaluation Model (Modelr): This model assesses how well a news item is supported by reliable sources. It incorporates a redistribution module and a relative time encoding mechanism to better capture the temporal relevance of background news.
  • Two-Hop Semantic Outlier Degree Evaluation Model (Modelu): This model quantifies how much a news item deviates semantically from its peers within the unreliable source background. It uses a novel approach to measure relative outlier degrees, mitigating the impact of tight semantic clusters.
  • Content Authenticity Evaluation Model (Modele): This model assesses the inherent truthfulness of the news content itself.
  • Multi-Model Mutual Benefit and Collaboration Framework: These three models are trained jointly, sharing a common bottom module. During inference, their predictions are combined to provide a more comprehensive and robust assessment of news authenticity. This framework enables multi-model generalization during training and multi-perspective prediction during inference.
  • Relative Time Encoding: A novel encoding method that captures the semantic relevance of news based on their release time gap.
  • Redistribution Module: Used in both Modelr and Modelu to center and rotate time-content vectors, optimizing early training checkpoints and expressing news relevance.

Datasets

The authors constructed a new English dataset for their experiments. This dataset includes:

  • Verified News: Scraped from fact-checking websites like Politifact and Snopes, covering January 2019 to October 2023. News is labeled as real or fake.
  • News with Source Reliability Label: Collected from the nela-gt2019-2022 datasets, comprising 33,212 reliable source news and 114,600 unreliable source news published from January 2019 to December 2022. Summarized texts were used.

This dataset is notable for including verified news and news with source reliability labels published within the same time period, addressing a gap in existing benchmarks.

Key Insights and Ideas

  • The Semantic Divide in Unreliable Sources: The core insight is that fake news creates a more cohesive semantic cluster within unreliable sources than real news does. This difference is a powerful signal for detection.
  • Dual-Source Reference is Crucial: Relying solely on reliable sources is insufficient. The interplay between reliable and unreliable source information provides a richer context for identifying fake news.
  • Temporal Relevance Matters: The time gap between news items is not just a timestamp; it encodes semantic relevance. The proposed relative time encoding effectively captures this.
  • Collaborative Models Enhance Robustness: Combining predictions from models focused on content, support from reliable sources, and outlier behavior in unreliable sources leads to a more accurate and resilient detection system.

Key Results

The paper reports strong performance across four experimental settings (Q1-Q4) designed to simulate real-world early detection scenarios. In the average performance across all experiments, RUSR achieved the highest macF1 score of 0.727, outperforming several baseline methods, including content-only approaches and those that incorporate evidence. Ablation studies demonstrated that removing any of the core models (Modelc, Modelr, Modelu) or key components like the relative time encoding led to a significant performance drop, highlighting the importance of each element. The multi-model collaboration was shown to be superior to any single model's performance.

Limitations and Future Work

  • Dataset Size: While the constructed dataset is novel, the paper notes that for some sub-models (e.g., Modelu), if the background news set (B) is too small, it becomes difficult to identify news as real or fake, leading to a default score of 0.5.
  • Computational Cost: The multi-model framework, while effective, likely incurs higher computational costs during training and inference compared to single-model approaches.
  • Future Work: The paper suggests further research into optimizing the redistribution module and exploring more advanced methods for capturing temporal relevance. The effectiveness of the multi-model collaboration framework also opens avenues for exploring more complex interaction patterns between models.

Summary updated

BibTeX
@inproceedings{feng2025contradicted,
  author = {Yifan Feng and Weimin Li and Yue Wang and Jingchao Wang and Fangfang Liu and Zhongming Han},
  title = {{Contradicted in Reliable, Replicated in Unreliable: Dual-Source Reference for Fake News Early Detection}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i22.34562},
}
Figure