Abstract
Deepfake detection faces significant challenges in cross-domain generalization and providing spatial explanations for forgery. This paper introduces Delocate, a two-stage framework designed for simultaneous detection and localization of tampered regions. The first stage, Recovering for Consistency Learning, utilizes a Masked Autoencoder (MAE) with a facial-part-guided masking strategy to learn the inherent consistency of real faces. The second stage, Localization for Discrepancy Learning, employs an encoder-decoder network and meta-learning to identify discrepancies between original and reconstructed faces, pinpointing manipulated areas. Experimental results across multiple benchmarks, including Celeb-DF and FaceForensics++, demonstrate that Delocate achieves state-of-the-art cross-domain performance, reaching an AUC of 91.3% on Celeb-DF, while providing robust spatial localization of forgery traces.
Introduction
As deepfake technology evolves, manipulated videos are becoming increasingly seamless, posing a major challenge for cross-domain detection. Most existing models are "black boxes" that provide a binary classification without explaining the spatial location of manipulations. Furthermore, localization methods often overfit to specific forgery patterns, making them ineffective against new, randomly located tampering traces. Delocate aims to create a reliable detector that is robust to unseen patterns and provides spatial explanations via precise localization.
Methodology
The Delocate framework adopts a two-stage approach to bridge the gap between detection and localization:
1. Recovering for Consistency Learning
The first stage employs a Masked Autoencoder (MAE) trained exclusively on authentic facial data. A novel masking strategy, guided by facial parts (eyes, nose, mouth) and inspired by the Facial Action Coding System (FACS), is used instead of random pixel masking. This ROI-based masking is more effective for deepfakes because manipulations are semantically tied to facial structures. Since the model is trained only on real faces, it becomes an expert in facial consistency. When presented with a fake face, the reconstruction fails in tampered areas because they lack the natural structural logic of a real human face.
2. Localization for Discrepancy Learning
The second stage takes the original face and its reconstruction from the first stage to map discrepancies. An encoder-decoder network identifies regions with high reconstruction error as likely tampered. To prevent the model from memorizing specific forgery artifacts, a meta-learning strategy is employed. By splitting data into meta-train and meta-test sets, the model is forced to learn the general task of finding discrepancies rather than specific fake patterns. This ensures that the framework remains effective against "unseen" deepfakes from unknown sources.
Experimental Evaluation
The model was evaluated on four major benchmark datasets: FaceForensics++ (FF++), Celeb-DF (CDF), DeeperForensics-1.0 (DFo), and the Deepfake Detection Challenge (DFDC).
Key Results
- Cross-Domain Performance: When trained on FF++ and tested on the high-quality Celeb-DF dataset, Delocate achieved an AUC of 91.3%, a significant improvement over previous localization-based methods (e.g., 86.2% by Chao et al.). On the DFo dataset, it reached a near-perfect 99.1% AUC.
- Robust Localization: Qualitative results demonstrate that Delocate produces cleaner, more accurate heatmaps of tampered regions compared to previous state-of-the-art methods, which often exhibited sporadic mismatches or noisy localization.
- Mutual Promotion: Learning to localize was found to improve classification accuracy. By forcing the network to identify the "where," it becomes much more confident in the "if" regarding forgery detection. This synergy between detection and localization is a core strength of the Delocate framework.
Discussion and Limitations
While Delocate excels in cross-dataset generalization, there is a minor trade-off in intra-dataset performance (a decrease of approximately 0.2% compared to some specialized models). Additionally, the two-stage process introduces more computational complexity than a single-pass classifier, though inference speeds remain commendable. Future work will explore deeper integration of temporal video-flow consistency to address frame-to-frame flickering in advanced deepfakes.
Conclusion
Delocate represents a significant step forward in deepfake detection by combining high-accuracy classification with explainable spatial localization. By focusing on the consistency of real faces rather than the artifacts of specific fakes, it achieves robust performance across diverse and unseen datasets.
Summary updated