Implicit Identity Driven Deepfake Face Swapping Detection

Baojin Huang, Zhongyuan Wang, Jifan Yang, Jiaxin Ai, Qin Zou, Qian Wang, Dengpan Ye

Abstract

Traditional deepfake detection methods primarily focus on identifying visual artifacts or manipulation traces. However, these methods often struggle to generalize across different forgery techniques. This paper introduces the Implicit Identity Driven (IID) framework, which shifts the focus toward face identity consistency. By analyzing the discrepancy between a face's "explicit identity" (visual appearance) and its "implicit identity" (underlying target identity), the proposed method achieves superior generalization performance on unseen datasets.

Introduction

The rapid advancement of face swapping technologies has created a significant challenge for digital forensics. Most existing detectors act as "artifact hunters," searching for specific algorithmic traces like blending glitches or frequency anomalies. While effective on known datasets, these models frequently fail when encountering novel manipulation methods—a phenomenon known as the "generalization gap."

The core insight of this work is that face swapping inherently involves an identity conflict. A forged face is a hybrid: it presents the explicit identity of a source person while often retaining subtle cues of the implicit identity of the original target. In a real face, these two identities are identical; in a fake face, they diverge. The IID framework exploits this fundamental discrepancy to build a robust, "fake-invariant" detector.

Conceptual overview of explicit vs. implicit identity distances in real and fake faces.

Methodology

The IID framework utilizes a CNN backbone (ResNet18) trained with two novel loss functions designed to map faces into a discriminative identity space.

Explicit Identity Contrast (EIC) Loss

The model employs a pre-trained face recognition model (CosFace) to extract explicit identity features. The EIC loss trains the backbone to ensure that for real faces, the learned features align closely with these explicit features. For forged faces, the loss pushes the learned features away from the explicit identity, highlighting the underlying forgery.

Implicit Identity Exploration (IIE) Loss

This loss focuses on extracting the "ghost" of the target identity remaining within the fake face.

  • Target-Aware Learning: For fakes with known targets, a margin-based classification groups fakes by their target identity.
  • Target-Agnostic Learning: For fakes with unknown targets, a lookup table maintains identity consistency across video frames.
  • Progressive Margin: A dynamic margin ($m_{fake}$) is used to stabilize training, first perfecting real identity alignment before increasing the difficulty for fake samples.
The IID framework architecture, illustrating the integration of Explicit and Implicit Embedding branches.

Experimental Evaluation

The researchers evaluated the IID framework across five major benchmarks: FaceForensics++ (FF++), Celeb-DF, FaceShifter, DeepFakeDetection (DFD), and the Deepfake Detection Challenge (DFDC).

Key Results

The primary strength of the IID framework lies in its cross-dataset generalization:

  • State-of-the-Art Generalization: When trained on FF++ and tested on the challenging DFDC dataset, IID achieved an AUC of 81.23%, outperforming the previous state-of-the-art (DCL) by 4.52%.
  • Robustness to Compression: On low-quality versions of Celeb-DF, the model maintained an AUC of 82.04%, significantly outperforming artifact-based methods that are sensitive to pixel-level degradation.
  • Ablation Study: Results confirmed that both EIC and IIE are necessary; IIE alone lacks the explicit reference point required to accurately quantify identity mismatch.

Discussion and Conclusion

By treating identity as a robust feature rather than searching for transient artifacts, the IID framework offers a more resilient approach to deepfake detection. Identity mismatch is inherent to the face-swapping process and cannot be easily erased by post-processing or improved blending.

However, limitations remain. The method is less effective for expression-only swaps (like Face2Face) where the identity remains constant. Additionally, while IID excels in generalization, artifact-based models may still perform better in "intra-dataset" scenarios where they can exploit specific local glitches. Future work involves integrating identity-driven features with local artifact detection to create a comprehensive detector that is both highly accurate and broadly generalizable.

Summary updated

Abstract

In this paper, we consider the face swapping detection from the perspective of face identity. Face swapping aims to replace the target face with the source face and generate the fake face that the human cannot distinguish between real and fake. We argue that the fake face contains the explicit identity and implicit identity, which respectively corresponds to the identity of the source face and target face during face swapping. Note that the explicit identities of faces can be extracted by regular face recognizers. Particularly, the implicit identity of real face is consistent with the its explicit identity. Thus the difference between explicit and implicit identity of face facilitates face swapping detection. Following this idea, we propose a novel implicit identity driven framework for face swapping detection. Specifically, we design an explicit identity contrast (EIC) loss and an implicit identity exploration (IIE) loss, which supervises a CNN backbone to embed face images into the implicit identity space. Under the guidance of EIC, real samples are pulled closer to their explicit identities, while fake samples are pushed away from their explicit identities. More-over, IIE is derived from the margin-based classification loss function, which encourages the fake faces with known target identities to enjoy intra-class compactness and inter-class diversity. Extensive experiments and visualizations on several datasets demonstrate the generalization of our method against the state-of-the-art counterparts.
BibTeX
@inproceedings{huang2023implicit,
  author = {Baojin Huang and Zhongyuan Wang and Jifan Yang and Jiaxin Ai and Qin Zou and Qian Wang and Dengpan Ye},
  title = {{Implicit Identity Driven Deepfake Face Swapping Detection}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2023},
  doi = {10.1109/cvpr52729.2023.00436},
}
Figure