Abstract
Deepfake generation methods are evolving rapidly, leaving behind unpredictable and random tampering traces that challenge standard detection models. This paper introduces Recap, a two-stage detection framework designed to enhance cross-domain generalization. Recap first reconstructs masked facial regions using a consistency-learning approach to expose inherent facial inconsistencies. It then maps these reconstructions to mathematically amplify the discrepancies between real and fake faces. By focusing on learning the structural consistency of real faces rather than specific artifacts, Recap achieves state-of-the-art performance across multiple benchmark datasets, including Celeb-DF and DFDC.
Introduction
As deepfake technology advances, the artifacts left behind by manipulation algorithms are becoming increasingly subtle and randomized. Traditional detection methods often function as specialized detectors, searching for specific, known forgery indicators such as unnatural blending boundaries or irregular blinking patterns.
However, modern deepfakes (e.g., Face2Face or FSGAN) alter random facial regions, leaving unpredictable traces. When a detector trained on one type of deepfake encounters a new, unseen manipulation method, it often fails—a phenomenon known as the cross-domain generalization problem. Recap addresses this by shifting the focus from "learning what is fake" to "learning what is real," identifying deepfakes by their lack of natural facial consistency.
Methodology
The Recap architecture is divided into two primary stages: Recovering for Consistency Learning and Mapping for Discrepancy Learning.
1. Recovering for Consistency Learning
Recap utilizes a Masked Autoencoder (MAE) based on Vision Transformers. Unlike standard MAE approaches that use random masking, Recap introduces an anatomy-aware masking strategy. Using facial landmarks, the model specifically masks Regions of Interest (ROIs) such as the eyes, nose, cheeks, and lips.
The autoencoder is trained exclusively on real faces. By masking 75% of these ROIs and requiring the model to reconstruct them, the network learns the inherent global consistency of human facial structures. When a fake face is processed, the decoder struggles to reconstruct it accurately because the manipulated regions lack this ground-truth consistency.
2. Mapping for Discrepancy Learning
The reconstructed faces are then passed through a ResNet-18 based mapping network. This stage acts as an amplifier, mapping well-reconstructed real faces to clean representations while forcing poorly reconstructed fake faces into even more distinct, "none-like" representations. This process artificially widens the mathematical gap between real and fake samples. To further improve robustness against unknown domains, the authors employ meta-learning during this stage, simulating cross-domain scenarios during training.
Experimental Results
The authors evaluated Recap using four benchmark datasets: FaceForensics++ (FF++), Celeb-DF, WildDeepfake (WildDF), and the Deepfake Detection Challenge (DFDC).
- Cross-Domain Generalization: When trained on FF++ and tested on unseen datasets (Celeb-DF, WildDF, DFDC), Recap achieved an average AUC of 85.5%, significantly outperforming other generalized methods like OST (79.4%) and MRL (77.1%).
- Intra-Domain Performance: On the FF++ dataset itself, Recap maintained a highly competitive average AUC of 99.0%.
- Ablation Studies: Research into masking ratios revealed that a 75% ratio is optimal. Lower ratios (55%) fail to force the model to learn robust features, while higher ratios (95%) make the reconstruction task too difficult even for real faces.
Conclusion and Future Work
Recap demonstrates that focusing on the structural consistency of real faces is a powerful strategy for detecting unpredictable deepfakes. By amplifying reconstruction errors through a mapping network, the model achieves superior generalization across diverse datasets.
Current limitations include a reliance on precise landmark detection (using dlib), which may be vulnerable to extreme head poses or heavy occlusions. Future research may explore making the pipeline more robust to landmark failures and extending the framework to better capture temporal consistencies across longer video sequences.
Summary updated