Multi-modal fake news detection: A comprehensive survey on deep learning technology, advances, and challenges

Jinna Lv, Yuan Gao, Li Li, Lei ShiJournal of King Saud University - Computer and Information Sciences 2025 · DOI

TL;DR

This comprehensive survey tracks the five-year evolution of multi-modal fake news detection, charting the field's progress from simple feature concatenation to sophisticated Large Language Model (LLM) reasoning. It provides a roadmap for how researchers are now integrating text, images, audio, and social propagation patterns to combat increasingly complex digital misinformation.

What Problem Does This Paper Address?

Fake news has evolved far beyond simple "clickbait" text. Today’s misinformation is a heterogeneous mix of doctored images, manipulated videos (deepfakes), and misleading audio, often spread through intricate social network webs.

The authors address the "heterogeneity gap": the fact that models designed for a single medium (like text-only BERT) are ill-suited for environments where the lie lives in the inconsistency between a caption and an image. The survey tackles three critical hurdles:

  1. Feature Fusion: How do we mathematically combine different data streams without losing vital information?
  2. Semantic Gaps: How do we ensure the model understands that a "peaceful" image paired with a "violent" caption is a red flag?
  3. Scalability: How do we detect these fakes in real-time as they go viral?

Methods

The paper categorizes the technical landscape into four major evolutionary stages:

  • General Content-based Fusion: The "early days" (pre-2021) focused on Early Fusion (merging features at the start), Late Fusion (averaging decisions from separate models), and Cross-modal Fusion (mapping features into a shared semantic space).
  • Attention Mechanisms: The shift from RNNs to Transformers. The authors highlight the move from Self-Attention (finding patterns within one modality) to Co-Attention (finding correlations between text and images).
  • Graph-based Interactions & Social Context: Instead of just looking at the content, these models look at the path. They use Graph Convolutional Networks (GCNs) to model how a story moves through a social network, identifying "spurious interactions" typical of bot-driven misinformation.
  • Knowledge Enhancement & LLMs: The current frontier. This involves "fact-checking" content against external databases (Wikipedia, Wikidata) or using LLMs (like GPT-4 or Llama) to perform logical reasoning, identifying contradictions that traditional pattern-recognition models might miss.

Datasets

The survey reviews 18 mainstream datasets. Key benchmarks include:

  • Weibo (2017): The most widely used (27% of studies), containing text, images, and social network data.
  • FakeNewsNet (PolitiFact & GossipCop): Rich in metadata and reader comments.
  • Fakeddit: A massive, fine-grained dataset with over a million samples across seven different categories of "fake" (e.g., irony, misleading association).
  • DGM4: A newer, more challenging dataset specifically focused on manipulated image-text pairs, designed to thwart simple detection.

Key Insights and Ideas

  • The "Entity" Advantage: High-performing models (like EMAF) succeed by focusing on entities (people, places, events). It is much harder for a bad actor to fake a consistent relationship between entities across multiple modalities than it is to simply doctor a single image.
  • Social Context as a "Truth Dimension": The way news propagates is often an independent indicator of its veracity. Fake news tends to have different structural diffusion patterns (e.g., "fake followers" or "irrelevant tweet clusters") compared to organic news.
  • From Recognition to Reasoning: We are moving away from models that ask, "Does this look like a fake image?" toward models that ask, "Does the evidence provided in this image logically support the claim made in the text?"

Key Results

The authors provide a head-to-head comparison of state-of-the-art (SOTA) models across major datasets:

  • On Weibo: The EMAF (Entity Matching) model leads with an impressive 97.4% accuracy.
  • On PolitiFact: CreditRank, which integrates user credibility assessment, hits 99% accuracy, proving that who shares the news is often as important as what is shared.
  • On Twitter: SGAMF (Sparse Gated Attention) demonstrates SOTA performance (95% accuracy) by effectively filtering out noise in multi-modal streams.

Limitations and Future Work

Despite the high accuracy scores in lab settings, the authors identify several "real-world" roadblocks:

  • The LLM Arms Race: While LLMs help detect fakes, they also make it trivial to generate high-quality, persuasive misinformation at scale.
  • Coarse Granularity: Most current research treats news as a binary (Real vs. Fake). In reality, misinformation is a spectrum (satire, partial truths, out-of-context media), and models need to become more nuanced.
  • The "Black Box" Problem: There is a dire need for interpretability. If a model flags a post as fake, it must be able to explain why (e.g., "The shadow in the image contradicts the timestamp of the text") for users to trust it.
  • Data Sparsity: Detecting fake news early—before it has a long propagation path—remains a significant technical bottleneck.

Summary updated

Abstract

The rapid development of social media and networks has dramatically accelerated the spread of multimodal fake news, posing significant threats to social stability, the public's right to information, and the credibility of news organizations. News modalities have advanced from simple text or images to combined video and audio formats. Integrating these heterogeneous data streams poses greater challenges for fake news detection. In recent years, the rapid advancement of Artificial Intelligence (AI) technology, including Large Language Models (LLMs), social network analysis, and external knowledge integration, has provided new insights and methods for detecting fake news. Nevertheless, there is a clear need for a thorough survey of multi-modal fake news detection. To address this, we have undertaken a comprehensive review to complement and consolidate existing research over the past five years. We systematically organize the key technologies and methods in this field and analyze the advantages of different technical approaches. Subsequently, we summarize mainstream fake news detection datasets and evaluation metrics, and analyze existing research directions. We discuss the challenges in this field, such as insufficient data annotation and model interpretability issues. Through this survey, we aim to provide researchers with a comprehensive understanding of fake news detection technologies and promote further development in this field.
BibTeX
@article{lv2025multi,
  author = {Jinna Lv and Yuan Gao and Li Li and Lei Shi and Siyu Li},
  title = {{Multi-modal fake news detection: A comprehensive survey on deep learning technology, advances, and challenges}},
  journal = {Journal of King Saud University - Computer and Information Sciences},
  year = {2025},
  doi = {10.1007/s44443-025-00317-7},
}
Figure