Abstract
Traditional deepfake detection methods primarily focus on identifying visual artifacts or manipulation traces. However, these methods often struggle to generalize across different forgery techniques. This paper introduces the Implicit Identity Driven (IID) framework, which shifts the focus toward face identity consistency. By analyzing the discrepancy between a face's "explicit identity" (visual appearance) and its "implicit identity" (underlying target identity), the proposed method achieves superior generalization performance on unseen datasets.
Introduction
The rapid advancement of face swapping technologies has created a significant challenge for digital forensics. Most existing detectors act as "artifact hunters," searching for specific algorithmic traces like blending glitches or frequency anomalies. While effective on known datasets, these models frequently fail when encountering novel manipulation methods—a phenomenon known as the "generalization gap."
The core insight of this work is that face swapping inherently involves an identity conflict. A forged face is a hybrid: it presents the explicit identity of a source person while often retaining subtle cues of the implicit identity of the original target. In a real face, these two identities are identical; in a fake face, they diverge. The IID framework exploits this fundamental discrepancy to build a robust, "fake-invariant" detector.
Methodology
The IID framework utilizes a CNN backbone (ResNet18) trained with two novel loss functions designed to map faces into a discriminative identity space.
Explicit Identity Contrast (EIC) Loss
The model employs a pre-trained face recognition model (CosFace) to extract explicit identity features. The EIC loss trains the backbone to ensure that for real faces, the learned features align closely with these explicit features. For forged faces, the loss pushes the learned features away from the explicit identity, highlighting the underlying forgery.
Implicit Identity Exploration (IIE) Loss
This loss focuses on extracting the "ghost" of the target identity remaining within the fake face.
- Target-Aware Learning: For fakes with known targets, a margin-based classification groups fakes by their target identity.
- Target-Agnostic Learning: For fakes with unknown targets, a lookup table maintains identity consistency across video frames.
- Progressive Margin: A dynamic margin ($m_{fake}$) is used to stabilize training, first perfecting real identity alignment before increasing the difficulty for fake samples.
Experimental Evaluation
The researchers evaluated the IID framework across five major benchmarks: FaceForensics++ (FF++), Celeb-DF, FaceShifter, DeepFakeDetection (DFD), and the Deepfake Detection Challenge (DFDC).
Key Results
The primary strength of the IID framework lies in its cross-dataset generalization:
- State-of-the-Art Generalization: When trained on FF++ and tested on the challenging DFDC dataset, IID achieved an AUC of 81.23%, outperforming the previous state-of-the-art (DCL) by 4.52%.
- Robustness to Compression: On low-quality versions of Celeb-DF, the model maintained an AUC of 82.04%, significantly outperforming artifact-based methods that are sensitive to pixel-level degradation.
- Ablation Study: Results confirmed that both EIC and IIE are necessary; IIE alone lacks the explicit reference point required to accurately quantify identity mismatch.
Discussion and Conclusion
By treating identity as a robust feature rather than searching for transient artifacts, the IID framework offers a more resilient approach to deepfake detection. Identity mismatch is inherent to the face-swapping process and cannot be easily erased by post-processing or improved blending.
However, limitations remain. The method is less effective for expression-only swaps (like Face2Face) where the identity remains constant. Additionally, while IID excels in generalization, artifact-based models may still perform better in "intra-dataset" scenarios where they can exploit specific local glitches. Future work involves integrating identity-driven features with local artifact detection to create a comprehensive detector that is both highly accurate and broadly generalizable.
Summary updated