Training-Free Multimodal Deepfake Detection via Graph Reasoning

Yuxin Liu, Fei Wang, Kun Li, Yiqi Nie, Junjie Chen, Yanyan Wei, Zhangling Duan, Zhaohong JiaarXiv 2025 · DOI · arXiv

Abstract

Large Vision-Language Models (LVLMs) often struggle with Multimodal Deepfake Detection (MDD) due to the subtlety of cross-modal inconsistencies. While fine-tuning offers a solution, it is computationally expensive and lacks generalization. This paper introduces GASP-ICL (Guided Adaptive Scorer and Propagation In-Context Learning), a training-free framework that treats deepfake detection as a sophisticated retrieval and in-context learning task. By utilizing a novel graph-based scoring mechanism (GSTAS), GASP-ICL identifies the most relevant "real vs. fake" examples to guide off-the-shelf LVLMs, significantly improving detection accuracy across various models without gradient updates.

Introduction

The rise of sophisticated generative AI has made Multimodal Deepfake Detection (MDD) a critical challenge. MDD requires identifying manipulations in images, text, or the semantic alignment between them. While state-of-the-art Large Vision-Language Models (LVLMs) like GPT-4o possess broad reasoning capabilities, they often overlook the minute artifacts characteristic of deepfakes.

Overview of the GASP-ICL framework compared to traditional detection methods.

The proposed GASP-ICL framework addresses these limitations by enhancing LVLMs through in-context learning (ICL). Instead of traditional fine-tuning, it leverages a smart "librarian" to provide the model with highly relevant reference cases, enabling frozen models to catch forgeries effectively.

The Challenge of Multimodal Deepfake Detection

Current approaches to MDD face three primary hurdles:

  1. Resource Intensity: Fine-tuning massive models for every emerging forgery type is computationally unsustainable.
  2. Subtle Cues: Deepfakes often contain logical "glitches" or tiny visual artifacts that general-purpose models are not inherently biased to detect.
  3. The Retrieval Gap: Standard ICL relies on similarity-based retrieval. However, simple semantic similarity often fails to distinguish between content-related samples and those that share structural forgery patterns.

Methodology: The GASP-ICL Framework

The core of GASP-ICL is a "knowledge bridge" built between a query sample and a library of known examples using graph theory.

Detailed architecture of the GASP-ICL framework, including multimodal retrieval, graph construction, and sample selection.

1. Multimodal Retrieval and Clustering

The process begins with a CLIP-based encoder, specifically fine-tuned for deepfake features, to extract candidates from a sample library. The system evaluates similarity across three domains: image-to-image, text-to-text, and joint image-text embeddings.

2. Graph Construction and Fusion

Rather than selecting the nearest neighbors in isolation, the framework constructs a Query-Centric Fused Graph. The query sample serves as the central node, connected to candidates based on modality-specific similarities. This structure captures the inter-relationships between samples, providing a richer context for reasoning.

3. GSTAS: Graph-Structured Taylor Adaptive Scorer

The technical centerpiece is the Graph-Structured Taylor Adaptive Scorer (GSTAS). This mechanism propagates activation signals from the query node across the graph. Using a Taylor-expanded gating mechanism, it weights these signals to "up-weight" samples sharing consistent manipulation patterns while suppressing noise. This allows for a tunable "receptive field" on the graph, enabling the model to learn from higher-order relationships.

4. In-Context Inference

The top-$k$ samples identified by GSTAS are formatted into a prompt and presented to a frozen LVLM. By observing these relevant examples, the model performs zero-shot classification with significantly higher precision.

Experimental Evaluation

The framework was evaluated on the DGM4 dataset, covering four manipulation categories: Text Swap, Face Swap, Text Attribute, and Face Attribute.

Key Results

Testing across seven LVLMs (including Qwen2.5-VL and LLaVA-v1.6) revealed:

  • Consistent Performance Gains: Qwen2.5-VL's accuracy improved from 49.71% (zero-shot) to 53.05% using GASP-ICL.
  • F1 Score Improvements: F1 scores frequently increased by over 10 percentage points, indicating a better balance between precision and recall.
  • Retrieval Efficacy: Using an adapted CLIP model for retrieval significantly outperformed standard frozen versions, highlighting the importance of forgery-aware feature extraction.

Conclusion

GASP-ICL demonstrates that sophisticated retrieval and graph-based reasoning can substitute for expensive model training in the domain of deepfake detection. While the method depends on the diversity of its sample library and introduces some inference latency, it provides a scalable and interpretable path forward for multimodal security. Future work will explore extending this graph reasoning to temporal inconsistencies in video deepfakes.

Summary updated

BibTeX
@misc{liu2025training,
  author = {Yuxin Liu and Fei Wang and Kun Li and Yiqi Nie and Junjie Chen and Yanyan Wei and Zhangling Duan and Zhaohong Jia},
  title = {{Training-Free Multimodal Deepfake Detection via Graph Reasoning}},
  year = {2025},
  eprint = {2509.21774},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.21774},
}
Figure