FaKnow: A Unified Library for Fake News Detection

Yiyuan Zhu, Yongjun Li, Jialiang Wang, Ming Gao, Jiali Wei

TL;DR

FaKnow is a comprehensive, PyTorch-based library designed to unify the fragmented landscape of fake news detection research. By providing standardized data structures and a suite of over 20 pre-implemented state-of-the-art models, it allows researchers to bypass boilerplate code and focus on reproducing results or developing new algorithms with ease.

What Problem Does This Paper Address?

The field of fake news detection has seen an explosion of deep learning models, but it suffers from a "Wild West" problem. Most models are developed in isolation using different frameworks, inconsistent data processing pipelines, and redundant codebases. This lack of standardization makes it incredibly difficult for researchers to reproduce previous findings or use existing models as baselines for new work.

Furthermore, fake news detection is inherently complex, often requiring the fusion of text, images (multi-modality), and social network structures (propagation graphs). Building these pipelines from scratch every time is labor-intensive and prone to error. FaKnow aims to be the "Scikit-learn" or "Hugging Face" specifically tailored for the fake news detection community.

Methods

The authors built FaKnow on a modular architecture consisting of three primary components:

  1. Data Module: Standardizes how data enters the system. It uses a generic JSON-based format for raw data and converts it into Python dictionaries for batch processing. A key feature here is lazy loading for multi-modal data; instead of loading thousands of images into memory at once, the library stores file paths and only loads pixels during the training traversal, optimizing the time-space tradeoff.
  2. Model Module: All models inherit from a unified AbstractModel class. This class extends PyTorch’s nn.Module by mandating two specific methods: calculate_loss (for training) and predict (for inference). This abstraction allows the trainer to interact with any model—whether it's a simple text classifier or a complex graph neural network—using the same interface.
  3. Trainer Module: This handles the "heavy lifting" of the machine learning lifecycle, including training loops, validation, testing, and auxiliary tasks like early stopping, gradient clipping, and visualization via TensorBoard.

The library categorizes models into two main camps: content-based (analyzing the news itself) and social context-based (analyzing how news spreads and who is sharing it).

Datasets

While the paper does not introduce a new dataset, it utilizes a massive array of existing benchmarks to validate the library’s reproducibility, including:

  • Weibo series: Weibo17, Weibo20, Weibo21, WeiboNEP.
  • Twitter series: Twitter15, Twitter16, TwitterMediaEval15/16.
  • Fact-checking sites: Politifact, FakeNewsNet.
  • Others: CED, MMCovid, PHEME, and TwitterMa.

Key Insights and Ideas

  • Unified Interface for Heterogeneous Tasks: The most significant "aha moment" is the realization that despite the vast differences between a TextCNN and a Graph Convolutional Network (GCN), their training and evaluation logic can be abstracted into a single workflow.
  • Standardization as a Reproducibility Tool: By re-implementing over 20 models within one framework, the authors identified that many "performance gaps" in literature stem from subtle differences in data preprocessing rather than the model architectures themselves.
  • Extensibility via API: The library is designed for growth. Researchers can integrate a new, complex model simply by overriding the init, forward, calculate_loss, and predict methods, inheriting all the logging and training infrastructure automatically.

Key Results

The authors performed extensive reproducibility experiments, comparing FaKnow’s implementations against the original papers' reported metrics.

  • High Fidelity: In the majority of cases, FaKnow results aligned within 1–2% of the original metrics.
  • Outperformance: Interestingly, for some models like ENDEF, CAFE, and GNNCL, the FaKnow implementations actually outperformed the original reported results by approximately 3%.
  • Consistency: For the MFAN model on the CED dataset, the accuracy disparity was a mere 0.001, demonstrating the library's precision in replicating experimental environments.

Limitations and Future Work

The authors acknowledge a few hurdles encountered during development:

  • Preprocessing Sensitivity: Some models, like SAFE and HMCAN, showed significant performance drops when the original authors' specific (and sometimes unavailable) preprocessing scripts weren't used. This highlights the ongoing need for standardized data cleaning in the field.
  • Unsupervised Complexity: Models relying on unsupervised modules (like the edge-consistency module in EBGCN) proved harder to replicate, as they are highly sensitive to latent relationship learning.

Future Work: The team plans to continually add new models as they emerge in top-tier conferences and develop further features to simplify the transition from raw social media scrapes to model-ready data.

Summary updated

BibTeX
@article{zhu2025faknow,
  author = {Yiyuan Zhu and Yongjun Li and Jialiang Wang and Ming Gao and Jiali Wei},
  title = {{FaKnow: A Unified Library for Fake News Detection}},
  journal = {Data Intell.},
  year = {2025},
  doi = {10.3724/2096-7004.di.2024.0026},
}
Figure