TwinD: Generalizing Deepfake Detection from Blending to Entire Face Synthesis

Andrii Yermakov, Jan Cech, Mario Fritz, Jiri MatasarXiv 2026 · arXiv

Deepfake detection is a cat-and-mouse game where the "mice" (generators) are evolving faster than the "cats" (detectors). A new paper from researchers at the Czech Technical University and CISPA Helmholtz Center introduces TwinD, a detector designed to end the trade-off between catching old-school "blending" fakes and modern "entire face synthesis" (EFS).

TL;DR

TwinD achieves state-of-the-art deepfake detection by training on "twins"—pseudo-fakes created from real images that mimic both blending artifacts and the subtle neural traces of AI decoders. It reaches a massive 96.3% mean AUROC across a benchmark of 19 datasets spanning 2019 to 2026, outperforming previous methods by nearly 9 percentage points.

What Problem Does This Paper Address?

Historically, deepfakes were "blending fakes": an AI-generated face was pasted onto a real person's head. Detectors got very good at finding the "seams" or statistical inconsistencies at the edges of these masks.

However, modern generators (like those used in Sora or Kling) often perform Entire Face Synthesis (EFS), generating the whole frame at once. Because there is no "pasting," there are no blending traces to find. Current facial deepfake detectors fail on EFS, while general AI-Generated Content (AIGC) detectors—which look for global "AI-ness"—fail on traditional blending fakes. TwinD is the first model to bridge this gap effectively.

Methods

The core innovation of TwinD is the use of "Twins": pseudo-fakes generated on-the-fly from real frames during training. This allows the model to learn the nature of a manipulation without needing a massive library of actual deepfakes.

  1. Reconstruction Twins: A real frame is passed through an autoencoder (specifically CogVideoX). The researchers then take the original real pixels and put them back into a random region of the reconstructed image. This teaches the model to recognize the subtle "neural traces" left by a decoder, even when they only cover part of the face.
  2. Blob Twins: This targets blending. It takes a real frame, alters it (changing contrast, hue, or downsampling), and blends it back into the original using a simple, smooth round "blob" mask. Unlike previous methods, it doesn't need facial landmarks or complex face-shaped masks.
  3. Architecture: TwinD uses a frozen PEcoreL (Perception Encoder) backbone. To keep training efficient, they use LoRA (Low-Rank Adaptation), fine-tuning only about 1.95% of the model's parameters.
  4. Paired Training: Crucially, every fake (whether a real deepfake or a twin) is paired with the exact real frame it was derived from. Both images receive identical augmentations (blur, JPEG compression, etc.), forcing the model to ignore "shortcuts" like image quality and focus purely on the manipulation artifacts.
Figure 1. Overview of TwinD. (a) A twin is a fake made from a real frame. A reconstruction twin is an autoencoder reconstruction with the real pixels put back inside a random grid or noise mask. A blob twin blends an altered copy of the frame with the frame itself through a blob mask. Right halves of the twins show the difference to the real frame. The traces of the autoencoder are amplified 10× for visualization. (b) Each fake is paired with the real frame it was made from. Both get the same augmentation, so they differ only by the manipulation. TwinD is a frozen PEcoreL with LoRA, trained on batches of such pairs. (c) Mean video-level AUROC (%) on the EFS and Blending datasets of Tab. 1. Gray bars show the best prior detector of each kind.

Datasets

The authors evaluated TwinD on a massive benchmark of 19 datasets released between 2019 and 2026. This is one of the most comprehensive evaluations in the field to date.

  • Training: FaceForensics++ (FF++) and Casual Conversations v2 (CCv2).
  • Evaluation Categories:
    • Blending: 10 datasets (e.g., DFD, FaceShifter, Celeb-DF-v2).
  • EFS: 5 datasets (e.g., TalkingHeadBench, OmniFake, DF26).
  • Mixed (B+EFS): 4 datasets (e.g., Celeb-DF++, RedFace).

Key Insights and Ideas

  • Decoder Traces are Universal: One of the most surprising findings is that training on the traces of a single autoencoder (CogVideoX) allows the model to detect fakes from dozens of other unseen generators. The "fingerprint" of a neural decoder is remarkably consistent across different architectures.
  • Identity is Irrelevant: Because twins are made from the same image, the "fake" and "real" have the same identity and expression. The fact that TwinD works so well proves that deepfake detection is primarily a low-level forensic task (finding pixel-level artifacts) rather than a high-level semantic one (finding "weird" expressions).
  • Twins are Enough: In an ablation study, a model trained only on twins (no actual deepfake datasets) still reached 94.3% AUROC, which is higher than any previous state-of-the-art model trained on real deepfakes.

Key Results

TwinD sets a new bar for generalization. As shown in the results table below, it dominates across all categories:

  • Overall Mean AUROC: 96.3% (The previous best, GenD-PE, was 87.4%).
  • EFS Performance: 95.6% (Previous best was only 78.1%).
  • Blending Performance: 98.1% (Previous best was 96.2%).
Table 1. Video-level AUROC (%) of SOTA methods on 19 datasets sorted by their macro-average (Avg). Above the dataset names are their release years. †A temporal model, which takes several frames as input in a single forward pass.

The model also shows strong robustness to common image degradations like JPEG compression and Gaussian blur, maintaining a lead over competitors even as image quality drops.

Figure 2. Robustness to image degradations. Mean video-level AUROC (%) over the 19 test datasets as the strength of each degradation increases. “Clean” is the undegraded benchmark of Tab. 1.

Limitations and Future Work

  • High-Frequency Masking: While robust to many things, TwinD’s performance dips under heavy Gaussian noise and extreme downscaling. These degradations likely "smear" the high-frequency decoder traces the model relies on.
  • Single Autoencoder: The current reconstruction twins rely on one autoencoder. The authors suggest that using a diverse ensemble of autoencoders for training could further improve robustness.
  • Temporal Cues: TwinD is currently a frame-level model. Incorporating temporal consistency (looking at how pixels change across frames) is a logical next step for video-specific detection.

Summary updated

Abstract

Facial deepfake detectors are usually trained on blending fakes, where a generated face is pasted back into a real frame. Recent generators synthesize the entire face and leave no traces of blending. On a benchmark of 19 datasets released from 2019 to 2026, the best prior facial deepfake detector reaches a mean AUROC of 96.2% on ten blending datasets, but none exceeds 78.1% on five datasets of entire face synthesis (EFS). AI-generated content (AIGC) detectors do not close this gap. We propose TwinD, a detector trained on FaceForensics++ fakes and two kinds of twins: fakes made from a real frame. A reconstruction twin is a real frame reconstructed by an autoencoder, with the real pixels put back in a random region. It targets the neural traces of a decoder. A blob twin blends an altered copy of a real frame with the frame itself, targeting blending artifacts. It needs no generator and no face-shaped mask. Together, these twins provide two complementary forensic cues that generalize to unseen manipulations. TwinDreaches a mean AUROC of 96.3%, 8.9 points above the prior state-of-the-art, with 98.1% on the blending datasets and 95.6% on the EFS datasets. Trained on twins alone, without a deepfake dataset, single models still reach 94.3% on average over three seeds. For reproducibility, the code, trained model, and training data are at: https://github.com/yermandy/TwinD
BibTeX
@misc{yermakov2026twind,
  author = {Andrii Yermakov and Jan Cech and Mario Fritz and Jiri Matas},
  title = {{TwinD: Generalizing Deepfake Detection from Blending to Entire Face Synthesis}},
  year = {2026},
  eprint = {2611.25935},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.25935},
}
Figure