A growing family of deepfake detectors now pairs a multimodal large language model (an LLM that reads images as well as text) with small specialist detectors. The specialists are compact convolutional networks that pick up low-level signals such as blending seams or frequency fingerprints. Their numerical scores are pasted into the language model's prompt, and the model writes a verdict and an explanation a human can read. This design fixes a real weakness, since language models cannot see sub-perceptual artifacts. It also creates a new one. From the language model's point of view, the injected score is a cue in the prompt. Work on LLM judges shows that such cues move verdicts substantially, and that judges then justify those verdicts with arguments about content while never mentioning the cue. An attacker who wants a fake to pass need not fool the large model. They only need to fool the small, often public specialist, which adversarial-perturbation work shows is easy. The language model would then write a fluent, plausible account of why the image is authentic.
We propose to test this "laundering" hypothesis directly and to test a structural fix. The fix keeps the language model blind to specialist scores. The model grades a short set of yes/no forensic checks from the pixels alone, and a small calibrated combiner merges the check probabilities with the specialist scores afterwards. Disagreement between the two channels is shown to the analyst instead of being smoothed over in prose. The core claim is that putting scores inside the prompt turns an explainable detector into a single point of failure whose failures are hidden by its own explanations. Separating the channels should keep in-distribution accuracy while making attacks on either channel both less effective and more visible.
What exists and where this sits
Several systems already put language models into deepfake detection, and they motivate this work rather than overlap with it. X2-DFD first measures which forgery features a language model detects reliably. For the features it misses, it trains external specific feature detectors and injects their predictions into the prompt during fine-tuning. It reports an average video-level AUC of 92.1% for its 13B version across datasets. It was not evaluated against adversarial attack, and its design is exactly the in-prompt fusion we study. EDVD-LLaMA likewise feeds hard landmark metrics into the prompt before the verdict. AuthGuard trains a language-guided encoder and concedes that its explanations are not guaranteed to be causally faithful. FakeVLM shows that training a model to explain artifacts raises accuracy by 37.4 points absolute. The multi-agent forensic system has four expert agents report to a judge agent. None of these papers asks whether the text channel launders an attack on the numeric channel, and none measures whether explanations acknowledge what actually drove the verdict.
The constraining evidence comes from the judge literature. The Silent Judge injects provenance and recency labels into pairwise judging prompts. It finds verdict shift rates up to +30% for GPT-4o and a cue acknowledgment rate of 0% in every condition. We treat a specialist score as the forensic analogue of an "EXPERT" label and reuse its two metrics.
MLLM-as-a-Judge shows that multimodal judges can hallucinate a flaw in a chain-of-thought step and then penalize it. It also shows they often judge better from text descriptions than from pixels. Both findings suggest that text cues will dominate weak visual evidence.
On the attack side, RAID shows that ensemble projected-gradient attacks drive white-box detector F1 to 0.0 and cut a strong detector's AUROC from 0.99 to 0.67 under black-box transfer. Black-box anti-forensics shows that a perturbation disguised as a sharpening mask fools detectors while looking cleaner to people.
The method for our fix comes from RocketEval. It breaks evaluation into independent binary checklist items graded by token probability, and it reports a 0.965 Spearman correlation with human preferences using a 2B judge on WildBench. We borrow its independent per-item grading. We do not take its claim that checklists improve accuracy on their own; we test that separately.
Research Program
The first part is a measurement study. We rebuild an in-prompt fusion detector in the X2-DFD style on an open multimodal backbone. Qwen2.5-VL-7B is the backbone used by the multi-agent system. The specialists are blending-boundary and frequency detectors, and the whole model is trained on FaceForensics++. We then run three kinds of probe.
The first probe is a counterfactual score swap. The image is left unchanged, and only the injected specialist numbers are replaced with values pointing the other way. This gives a verdict shift rate attributable purely to the score channel. We also measure the acknowledgment rate, meaning the fraction of shifted explanations that mention the specialist score rather than citing visual artifacts.
The second probe is a realistic attack on the score channel. We apply projected-gradient perturbations computed only against the small specialists, never against the language model. Then we measure end-to-end detection.
The third probe tests explanation faithfulness. For artifacts the model names in a specific face region, we inpaint that region with content from the source real frame and check whether the verdict actually changes.
The hypothesis is falsifiable in a clear way. It fails if score swaps shift few verdicts, or if shifted explanations routinely cite the score.
The second part is the separated design. A strong model writes a fixed bank of ten to fifteen forensic checks for face images. Examples are "the jawline boundary shows a color discontinuity" and "specular highlights in the two eyes disagree in direction." The blind language model is fine-tuned to answer each check independently, and we read off the probability of "yes." A logistic combiner is fit on held-out training-distribution data. It takes the check probabilities and the specialist scores and outputs the verdict. A disagreement statistic flags inputs where the specialists say "real" but several checks fire with high confidence, or the reverse.
The explanation shown to the analyst is assembled only from checks that fired, together with the specialist scores, each labeled with its source. Because nothing else is shown, the explanation cannot cite evidence the decision did not use.
Baselines
The naive reference point is an XceptionNet classifier trained on FaceForensics++, following FaceForensics++. It is the default practitioners deploy. Its known collapse to roughly 56.9% average AUC across detectors on Celeb-DF (Celeb-DF) sets the floor for generalization. Beating it carries no information beyond confirming that the pipeline is wired correctly.
The strongest existing method is in-prompt fusion as in X2-DFD, at the reported 92.1% average video-level AUC. We will reimplement it at matched backbone size, with the same specialists, the same FaceForensics++ training data, and the same fine-tuning budget in GPU-hours and hyperparameter trials as our separated design. Where we can, we also evaluate released X2-DFD weights to confirm that our reimplementation lands near the reported figure.
Our design beats this baseline if two conditions hold together. First, its clean cross-dataset AUC is within two points of the baseline. Second, under specialist-only attacks it retains substantially more AUC and flags a majority of the successfully attacked fakes through its disagreement statistic.
Two further reference points come from the file. The multi-agent system reports 53.28% F1 on unseen generators in FaceVid-Forensics-100K against 46.22% for zero-shot Gemini-3.5-Flash. We report zero-shot frontier prompting as an additional baseline. No reported result was found on Lacuna for any language-model-based deepfake detector under RAID-style attacks, so those numbers will be new.
The ablation that isolates the key ingredient is a version where the specialist scores sit in the prompt but the model must still answer the checklist. This separates "checklist format" from "the language model never sees the score." A second ablation replaces the checklist with a single blind verdict probability, which isolates the value of item-level evidence.
Why not just…?
Why not just adversarially train the specialist detectors? This hardens one channel but leaves the laundering structure intact. Adversarial training against one perturbation family is routinely broken by another, and RAID's transfer results show that ensembles of perturbations generalize across architectures. We will include an adversarially trained specialist arm inside in-prompt fusion. If its attacked accuracy matches our separated design, and its explanations acknowledge score dependence, the structural argument weakens. We will report that outcome as it falls.
Why not just late-fuse a blind verdict with the detector score, without any checklist? This is the real threat to the second half of the proposal. Most of the robustness gain may come from blindness alone, and our single-verdict ablation tests exactly that. The checklist earns its place only if two things hold. Its item probabilities must give a disagreement statistic that catches attacked inputs at a higher rate, at a fixed false-flag rate on clean data, than the scalar disagreement does. Its explanations must also pass the inpainting faithfulness test more often. If neither holds, we will recommend plain blind late fusion and present the checklist as a negative result.
Why not just instruct the judge to discount or report its reliance on the scores? The Silent Judge found that judges never acknowledged the cues that shifted their verdicts. That makes self-report a weak safeguard, and MLLM-as-a-Judge shows that reasoning prompts can make hallucinated evidence worse. We will still run an "ignore scores unless visually corroborated" prompt arm and measure its shift and acknowledgment rates directly, rather than assume it fails.
Proposed Experiments
Training uses FaceForensics++ at high-quality compression. Cross-dataset testing uses Celeb-DF, DFDC, and DF40, with frame scores averaged into video-level AUC. Adversarial testing uses RAID's attacked images (dataset, code) at its three budgets. Because RAID's images are general synthetic images rather than faces, we retrain the specialists on its $D^3$ source images for that arm. We also generate our own specialist-only attacks on face frames from Celeb-DF using the RAID codebase.
The metrics are AUC and F1, verdict shift rate and acknowledgment rate under score swaps, attack success rate, the disagreement statistic's detection rate at a 5% false-flag rate on clean data, and the inpainting faithfulness rate. A human study on a sample of explanations asks forensic-literate raters to judge whether each explanation would have warned them that something was off on attacked inputs.
Three results would decide the question. The first is whether score swaps on unchanged images flip a large share of in-prompt fusion verdicts while acknowledgment stays near zero. That outcome would confirm the laundering phenomenon in the forensic setting. The second is whether specialist-only perturbations, which never touch the language model's gradients, collapse in-prompt fusion while the separated design both degrades less and flags the attacked fakes. The third is whether the checklist beats blind scalar late fusion on detection of attacks and on faithfulness. If it does not, the contribution reduces to the simpler and still useful finding that the language model should never see the specialist's number.
References
- X2-DFD: A framework for explainable and extendable Deepfake Detection
- EDVD-LLaMA: Explainable Deepfake Video Detection via Multimodal Large Language Model Reasoning
- AuthGuard: Generalizable Deepfake Detection via Language Guidance
- Spot the Fake: Large Multimodal Model-Based Synthetic Image Detection with Artifact Explanation
- Multi-Agent Forensic Reasoning for Generalizable Deepfake Video Detection
- The Silent Judge: Unacknowledged Shortcut Bias in LLM-as-a-Judge
- MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark
- RAID: A Dataset for Testing the Adversarial Robustness of AI-Generated Image Detectors
- Synthesizing Black-Box Anti-Forensics Deepfakes With High Visual Quality
- RocketEval: Efficient automated LLM evaluation via grading checklist
- FaceForensics++: Learning to Detect Manipulated Facial Images
- Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics