Imagine a courtroom where an expert witness points to a video and says, "This is a deepfake. I am 98% sure." When asked why, they reply, "Because my neural network outputted a high probability score."
In legal, journalistic, and forensic contexts, this answer is insufficient. As Siwei Lyu argues in Deepfake Detection: Current Challenges and Next Steps, numerical probability scores lack the explanatory power required for accountability and validation in high-stakes environments. Traditional deepfake detection models operate as black boxes, identifying statistical or frequency-domain anomalies that are invisible and uninterpretable to humans (Leandro A. Passos et al.; Thanh Nguyen 0004 et al.). This lack of transparency makes it impossible for human operators to verify alerts or learn to spot manipulation cues themselves.
To bridge this gap, a major research direction has shifted from simple binary classification toward explainable deepfake detection. Instead of merely asking "Is this image real?", these systems answer "Where is it manipulated?" and "Why is it considered a forgery?"
Grounding Forgeries in Space and Time
To make a detection verifiable, a model must first localize its evidence. If a model claims a video is fake, it should pinpoint the exact pixels or frames containing the artifact.
Early approaches to localization relied on attention mechanisms. For example, Hao Dang et al. introduced an attention layer that forces the network to produce a heatmap highlighting the specific regions of a face that have been tampered with. While helpful, heatmaps can still be vague. To provide more structured, case-based reasoning, Loc Trinh et al. developed DPNet, which compares video motion against a learned set of "dynamic prototypes"—representative examples of unnatural temporal artifacts. By showing a human operator exactly which prototype of unnatural motion a video resembles, the system provides a clear precedent for its decision. Similarly, Yi Li demonstrated that measuring similarity to prototypes of real data can yield highly efficient, interpretable classifications.
The current state-of-the-art moves beyond coarse heatmaps to pixel-level segmentation. The framework proposed by Lixiong Qin et al. (FiFa) generates precise pixel masks of forgery artifacts interleaved directly with natural language descriptions. This ensures that anatomical references—such as a misaligned jawline or blending boundary—are explicitly grounded in spatial coordinates.
Translating Pixels to Paragraphs via Vision-Language Models
While spatial masks tell us where an artifact is, they do not explain what it means. To make explanations truly human-readable, researchers have integrated Multimodal Large Language Models (MLLMs) to translate low-level statistical anomalies into high-level semantic reasoning.
The core mechanism relies on a two-stage pipeline: a specialized vision encoder extracts forensic features from the image, and a language model translates those features into a text narrative. For instance, Xiao Guo et al. developed M2F2-Det, which combines CLIP's visual-linguistic features with an LLM using a forgery-specific prompt learning strategy to generate plain-English justifications.
However, standard LLMs lack the domain-specific "forensic sense" required to identify subtle deepfake artifacts. To overcome this, Peipeng Yu et al. and Peipeng Yu et al. introduced frameworks that use specialized prompt learners to map technical forensic artifacts into a linguistic space the LLM can comprehend, allowing the model to leverage its broad common-sense reasoning to evaluate the authenticity of an image.
A major risk of using language models for forensics is hallucination—the model might generate a plausible-sounding explanation that does not match the actual visual evidence. To enforce strict alignment, Tuan Nguyen et al. developed Paragraph-level Policy Optimization (PRPO). This reinforcement learning framework penalizes the model if its textual arguments diverge from the actual visual artifacts present in the image, forcing the MLLM to ground its reasoning strictly in physical reality.
Interactive Forensics and Visual Question Answering
In practice, a single static explanation is rarely enough. A forensic investigator might want to ask follow-up questions about a specific facial feature or double-check a suspicious region. This has led to the reframing of deepfake detection as an interactive Visual Question Answering (VQA) task.
Niki Maria Foteinopoulou et al. established a benchmarking protocol that treats face forgery detection as a fine-grained VQA problem, allowing users to query models about specific semantic anomalies. To enable fluid, multi-turn dialogues, Lixiong Qin et al. developed DFA-GPT, a specialized MLLM capable of engaging in conversational deepfake analysis, explaining complex artifacts like phoneme-to-lip mismatches or blending boundaries upon request. For non-expert users, Shahroz Tariq et al. built a framework that synthesizes visual heatmaps, automated image captioning, and LLM refinement into a cohesive interactive dashboard, making deepfake verification accessible to journalists and legal professionals.
Inherent Interpretability via Physical and Behavioral Constraints
While post-hoc explanations make deep neural networks more transparent, another branch of explainable forensics bypasses deep learning black boxes entirely by focusing on inherently interpretable physical or behavioral laws.
Generative models are excellent at matching statistical distributions, but they often fail to respect the underlying physics of the real world. For example, Shu Hu et al. developed a method to expose GAN-generated faces by analyzing corneal specular highlights—the tiny reflections of the environment visible in human eyes. Because GANs generate each eye relatively independently, the reflections in the left and right eyes are almost always geometrically inconsistent. This physical mismatch provides an unforgeable, human-verifiable proof of manipulation.
Similarly, instead of passively analyzing static artifacts, detectors can actively probe the media. Govind Mittal et al. introduced GOTCHA, a real-time challenge-response system for live video calls. By forcing a user to perform specific interactive actions (like passing a hand in front of their face), the system exploits the structural latency and occlusion failures of generative pipelines. The resulting visual glitches provide immediate, self-explanatory evidence of a deepfake to both human observers and automated systems.
Key Foundational Work
Interpretable and Trustworthy Deepfake Detection via Dynamic Prototypes (2021) — contributed 1 concept elements to this direction.
Exposing GAN-Generated Faces Using Inconsistent Corneal Specular Highlights (2021) — contributed 1 concept elements to this direction.
Finding Facial Forgery Artifacts With Parts-Based Detectors (2021) — contributed 1 concept elements to this direction.