Introduction
Deepfake detection has become a critical area of research as face manipulation techniques grow increasingly sophisticated. While existing detectors perform well on datasets they were trained on, they often fail to generalize to "unseen" domains—new manipulation methods or datasets with different characteristics. This lack of robustness stems from models overfitting to domain-specific artifacts rather than learning universal features of forgery.
To address this, we present Learning-to-Weight (LTW), a meta-learning framework designed to enhance domain generalization. By automatically assigning importance to training samples based on their contribution to generalization, LTW builds a more robust detector capable of identifying novel forgeries.
The Learning-to-Weight (LTW) Framework
The core challenge in face forgery detection is the "Deepfake arms race," where attackers constantly invent new methods like identity swapping, expression transfer, or neural textures. Standard models often capture the specific "fingerprints" of these generators, leading to poor performance on unseen data.
The LTW framework addresses this by treating training as a meta-learning problem. It consists of a Detector Branch (a binary classifier, typically using EfficientNet-b0) and a Weight-Aware Branch. The latter is a lightweight network that predicts an importance score (weight) for each training sample based on its latent features.
Meta-Split Strategy
To simulate the encounter with unseen domains during training, LTW employs a Meta-Split Strategy. Multi-source training data is randomly partitioned into a "training set" and a "meta-test set." This allows the model to "practice" generalization by evaluating how updates performed on the training set affect performance on the meta-test set.
Meta-Optimization and Loss Functions
The training process involves a sophisticated meta-optimization loop. First, the model performs a "virtual" update on the training set. It then evaluates the loss on the meta-test set. If a particular training sample helps the model perform better on the "unseen" meta-test set, the weight-aware branch learns to assign higher weights to similar samples in future iterations. This effectively teaches the model to ignore domain-specific noise and focus on domain-invariant features.
Beyond standard cross-entropy (CE) loss, LTW introduces the Intra-Class Compact (ICC) Loss. This loss function is based on the observation that "real faces" form a stable, consistent category, whereas "fake faces" are infinitely diverse. ICC loss pulls all real samples toward a single center in the feature space while pushing fake samples away, creating a tight, reliable boundary for authenticity.
Experimental Evaluation
The LTW method was evaluated against prominent benchmarks, including FaceForensics++ (FF++), Celeb-DF, and the Deepfake Detection Challenge (DFDC). Two primary setups were tested:
- General Intra-Datasets (GID): Training on three manipulation types within FF++ and testing on the fourth.
- General Cross-Datasets (GCD): Training on FF++ and testing on entirely different datasets like Celeb-DF and DFDC.
Key Results
LTW consistently outperformed state-of-the-art baselines such as MLDG and ForensicTransfer. Key findings include:
- Superior Generalization: LTW showed an average improvement of approximately 2% across all benchmarks.
- Robustness to Compression: On low-quality, compressed video (GID-F2F40), LTW achieved an AUC of 0.724, significantly higher than the baseline's 0.537.
- Cross-Dataset Success: When tested on the challenging Celeb-DF dataset after training on FF++, LTW reached an accuracy of 63.4%, demonstrating its effectiveness in real-world, unseen scenarios.
Conclusion and Future Work
By learning what to ignore, the LTW framework shifts the focus from learning specific forgery artifacts to identifying universal characteristics of real and fake faces. While meta-learning introduces higher computational costs during training due to second-order derivatives, the resulting models are significantly more robust. Future research may explore the resilience of this weighting strategy against adversarial attacks specifically designed to bypass deepfake detectors.
Summary updated