Abstract
Recent advances in deepfake forensics have primarily focused on improving classification accuracy and generalization. However, existing algorithms often lack intuitive interpretability and identity traceability, which are crucial for forensic investigations. We introduce the DeepFake Identity Recovery scheme (DFREC), a novel framework designed to recover both source and target faces from a single deepfake image. DFREC utilizes an Identity Segmentation Module (ISM), a Source Identity Reconstruction Module (SIRM), and a Target Identity Reconstruction Module (TIRM) featuring an identity-aware Masked Autoencoder. Our evaluations across six high-fidelity face-swapping attacks demonstrate that DFREC achieves superior recovery performance and high fidelity compared to state-of-the-art methods.
Introduction
The rise of sophisticated deepfake technology has created a pressing need for advanced forensic tools. While current methods excel at detecting whether an image is manipulated, they rarely provide insights into the identities involved. For forensic investigators, identifying the "source" (the person providing facial features) and the "target" (the original person in the video) is essential for establishing a chain of evidence. DFREC addresses this by shifting the focus from binary detection to comprehensive identity tracing, allowing for the restoration of original identities involved in a face-swap.
Methodology
The DFREC framework operates through three specialized modules that work in sequence to decouple and reconstruct identities.
Identity Segmentation Module (ISM)
The ISM employs a DeepLabv3 architecture to generate a probability map of the forged image. This map distinguishes between pixels belonging to the "source" identity (the swap) and the "target" background context, providing a spatial guide for subsequent reconstruction.
Source Identity Reconstruction Module (SIRM)
Using the segmented source information, the SIRM reconstructs the source face. A key insight of this module is that the forged region often retains "latent target identity features"—subtle cues of the original target's identity that persist despite the swap due to the inherent correlation between identity and attributes like pose and expression. The SIRM extracts these features to aid in target reconstruction.
Target Identity Reconstruction Module (TIRM)
The TIRM is the core of the recovery process, utilizing an Identity-aware Masked Autoencoder (MAE):
- Semantic-Guided Masking: Unlike standard MAEs that mask random patches, DFREC uses the ISM's probability map to specifically mask out the forged source patches.
- Identity Fusion: A Vision Transformer (ViT) encoder processes the unmasked background, while an Identity Fusion Decoder merges these features with the latent target features from the SIRM to "inpaint" the original target face with high fidelity.
The model is optimized using a quadruple loss function encompassing Identity Loss (FaceNet), Perceptual Loss (VGG), Attribute Loss, and Patch Recovery Loss.
Experimental Evaluation
DFREC was evaluated on three major benchmarks: FaceForensics++, CelebaMegaFS, and FFHQ-E4S, covering various forgery algorithms such as DeepFake (DF), FaceShifter (FShi), and IDInjection (IDI).
Performance Analysis
DFREC significantly outperforms state-of-the-art methods like DFI and Delocate. On the CelebaMegaFS (FTM) dataset, DFREC achieved an Identity Similarity (IDSim) score of 0.7192, nearly doubling the performance of DFI (0.4137).
In terms of image quality, DFREC achieved a Frechet Inception Distance (FID) of 8.34 on the FFHQ-E4S dataset, outperforming the next best method, MAT (9.82). The framework also demonstrates robust generalization, maintaining high performance even on unseen forgery types. In intra-dataset testing, DFREC achieved near-perfect identity recovery accuracy ($Acc_{id}$) ranging from 98% to 100%.
Conclusion
DFREC proves that treating target face recovery as an informed inpainting task effectively avoids "identity leakage"—a common issue where source features persist in the recovered target face. Furthermore, the ability to recover pristine faces provides an interpretable signal for forgery detection: if the recovered identities do not align with the input, the image is likely a deepfake. Future research will focus on enhancing robustness to "in-the-wild" conditions, such as extreme poses or poor lighting, and integrating these capabilities into real-time forensic pipelines.
Summary updated