Diff-ID: An Explainable Identity Difference Quantification Framework for DeepFake Detection

Chuer Yu, Xuhong Zhang, Yuxuan Duan, Senbo Yan, Zonghui Wang, Yang Xiang, Shouling Ji, Wenzhi Chen

DeepFake detection has become a high-stakes arms race. While modern classifiers can spot known forgery techniques with near-perfect accuracy, they often crumble when faced with a new, unseen manipulation method or heavy image compression. A new paper, "Diff-ID: An Explainable Identity Difference Quantification Framework for DeepFake Detection," proposes a shift in strategy: instead of looking for technical "glitches" or artifacts, why not look at the person's identity itself?

TL;DR

Diff-ID detects DeepFakes by quantifying the subtle "identity loss" that inevitably occurs during face-swapping. By using a face-swapping generator to align a suspected image with an authentic reference image in the same attribute space (pose, expression, lighting), the framework isolates and measures identity discrepancies, achieving superior generalization and robustness to compression.

What Problem Does This Paper Address?

Most DeepFake detectors are trained to find low-level artifacts—things like blending boundaries, strange textures, or inconsistent head poses. However, these "forgery traces" are fragile. New synthesis methods (like HeadSwapper) leave fewer traces, and simple JPEG compression can wipe them out entirely.

Furthermore, these models tend to overfit. A detector trained on the Deepfakes dataset might achieve 95% accuracy there but drop to 65% when tested on the CelebDF dataset. The authors argue that we need a high-level, semantic feature that is universal across all face-swapping methods. Their candidate? Identity inconsistency.

Methods

The core insight of Diff-ID is that face-swapping is imperfect. Because facial identity features (like the shape of a nose) are "entangled" with attribute features (like a smile or a head tilt), the process of fusing a source identity into a target face inevitably causes a non-negligible loss of the original identity.

To measure this, Diff-ID uses a Reference-Assisted approach:

  1. Image Reconstruction Module: Given a test image ($I_{test}$) and an authentic reference image ($I_{ref}$) of the same person, the system uses a face-swapping generator (the authors used SimSwap) to align them. It generates versions of the faces where the identity of one is mapped onto the attributes (pose, expression, etc.) of the other.
  2. Identity Difference Quantification: By aligning both images to the exact same "attribute space," the researchers can isolate the differences caused purely by identity. They calculate the pixel-level $L_2$ distance and the angular distance (cosine similarity) between these aligned pairs.
  3. Normalization: To ensure the metric isn't just picking up on the generator's own reconstruction errors, they divide the identity loss by the reconstruction loss.
  4. Adaptive Fine-tuning: The authors fine-tune the underlying generator with specific "identity" and "attribute" constraints, making it more sensitive to subtle identity changes while better preserving background and lighting.

Datasets

The researchers evaluated Diff-ID across four major benchmarks:

  • FaceForensics++ (FF++): The standard baseline for forgery detection.
  • Google DeepFake Detection (DFD): Contains diverse scenes and poses.
  • CelebDF: A challenging dataset with high-quality, improved synthesis.
  • DeeperForensics-1.0 (DFo): Focuses on professional-grade manipulations.

For training (fine-tuning the generator), they used only 1,000 images from the CelebA dataset, notably without needing any fake images for training.

Key Insights and Ideas

  • Alignment is Everything: Simply comparing identity embeddings (like ArcFace) of two different photos is too noisy because pose and lighting interfere. By using a generator to "force" both images into the same pose and lighting first, the identity comparison becomes much more precise.
  • Explainability: Unlike "black-box" CNNs, Diff-ID is explainable. It produces "diff" images that visually highlight where the identity is failing—often around the eyes, eyebrows, and mouth corners.
  • Robustness to Distortion: Because identity is a high-level semantic feature, it survives image compression much better than low-level pixel artifacts. Even at low JPEG quality, the "identity" of the face remains detectable.

Key Results

Diff-ID demonstrated remarkable generalization—the ability to work on datasets it wasn't specifically tuned for:

  • Superior Generalization: While a standard Xception-based detector dropped to 59.4% AUC on the CelebDF dataset, Diff-ID maintained a high 91.1% AUC.
  • State-of-the-Art Performance: On the DFD and DFo datasets, Diff-ID achieved 98.5% and 98.3% AUC respectively, outperforming most existing frame-level and video-level methods.
  • Compression Resilience: In tests with heavy JPEG compression (Quality Factor = 20), the performance drop was minimal (less than 2% AUC), whereas traditional artifact-based detectors often fail completely in these scenarios.

Limitations and Future Work

  • The "Reference" Requirement: The biggest hurdle is that Diff-ID requires an authentic reference image of the person the test image claims to be. While practical for protecting celebrities or users on ID-verified platforms (like LinkedIn or banking apps), it is less useful for detecting "random" fakes of unknown people.
  • Generator Dependency: The framework's accuracy is tied to the quality of the face-swapping generator used for alignment. If the generator itself is too "lossy," it introduces noise into the metric.
  • Forgery Types: The current work focuses on face-swapping. The authors acknowledge that "face reenactment" (where only expressions are changed, not the identity) results in much lower identity loss and remains a challenge for this specific approach.

Summary updated

Abstract

In recent years, DeepFake technologies have seen widespread adoption in various domains, including entertainment and film production. However, they have also been maliciously employed for disseminating false information and engaging in video fraud. Existing detection methods often experience significant performance degradation when confronted with unknown forgeries or exhibit limitations when dealing with low-quality images. To address this challenge, we introduce Diff-ID , a novel approach designed to elucidate and quantify the identity loss induced by facial manipulations. When assessing the authenticity of an image, Diff-ID leverages a genuine image of the same individual as a reference and processes two images jointly. It aligns the reference image and the test image into the same identity-insensitive attribute feature space using a face-swapping generator. This alignment allows us to observe the identity disparities between the two images through the differences in the aligned generation pairs. Subsequently, we have developed a custom metric designed to quantify the identity loss relative to the reference image in the test image. This metric effectively distinguishes forgery images from the real ones. Extensive experiments have demonstrated the exceptional performance of our approach. It achieves a high level of detection accuracy on DeepFake images and showcases state-of-the-art generalization capabilities when confronted with previously unknown forgery methods. Moreover, it exhibits robustness even in the presence of image distortions.
BibTeX
@article{yu2024diff,
  author = {Chuer Yu and Xuhong Zhang and Yuxuan Duan and Senbo Yan and Zonghui Wang and Yang Xiang and Shouling Ji and Wenzhi Chen},
  title = {{Diff-ID: An Explainable Identity Difference Quantification Framework for DeepFake Detection}},
  journal = {IEEE Trans. Dependable Secur. Comput.},
  year = {2024},
  doi = {10.1109/tdsc.2024.3364679},
}

Loading related content...

Figure