Abstract
Deepfake detection has traditionally relied on spatial artifacts within individual frames. However, as generative models improve, these spatial cues become less reliable. This paper presents the Temporal Dropout 3-Dimensional Convolutional Neural Network (TD-3DCNN), a framework that shifts the focus to temporal inconsistencies. By employing a novel Temporal Dropout sampling strategy and 3D Inception modules, the model achieves state-of-the-art performance on major benchmarks, demonstrating significant robustness and generalization capabilities.
Introduction
The proliferation of photorealistic deepfakes poses a significant challenge to digital integrity. While modern forgeries often exhibit near-perfect spatial quality, they frequently lack physiological consistency across time—manifesting as unnatural blinking, jittery movements, or inconsistent facial expressions. Existing "image-level" detectors often fail when confronted with high-quality synthesis or unseen datasets because they ignore these temporal dynamics.
The authors argue that the key to robust detection lies in the temporal dimension. The challenge is to train 3D models effectively without overfitting to specific sampling patterns or incurring prohibitive computational costs.
Methodology
The proposed TD-3DCNN framework introduces two major architectural innovations designed to capture multi-scale temporal features.
3D Inception Modules
To extract features across both space and time, the network utilizes 3D Inception modules. These modules employ 3D kernels of varying sizes (e.g., $3\times3\times3$ and $5\times5\times5$), allowing the model to capture fine-grained temporal details (like micro-expressions) alongside broader motion patterns (like head rotations) simultaneously.
Temporal Dropout (TD)
The core innovation is the Temporal Dropout sampling strategy. During training, instead of using continuous frame sequences, the model selects a raw sequence and then randomly "drops" a portion of the frames. This acts as a powerful data augmentation technique, forcing the network to learn robust representations from incomplete data. This prevents the model from developing "sampling biases" and improves its ability to generalize to videos with varying frame rates or quality degradations.
Experimental Results
The TD-3DCNN was evaluated on three challenging benchmarks: Celeb-DF(v2), DFDC, and FaceForensics++.
- Celeb-DF(v2): The model achieved an AUC of 88.83%, marking a 32.63% average improvement over six other state-of-the-art detectors, including MesoNet and Two-stream NN.
- DFDC: On the massive Deepfake Detection Challenge dataset, the model reached an AUC of 78.97%, outperforming the next best method by over 3%.
- Robustness: Stability tests confirmed that the TD method significantly outperforms standard systematic or continuous sampling in maintaining consistent predictions across a video.
Discussion and Conclusion
The results demonstrate that the "temporal heartbeat" of a video is a highly reliable signal for deepfake detection. By intentionally introducing "incompleteness" during training via Temporal Dropout, the model becomes more adept at identifying the underlying physical inconsistencies of forged content. While 3DCNNs are computationally demanding, their superior generalization—especially in cross-dataset scenarios—makes them a vital tool in the evolving landscape of deepfake detection. Future work will explore optimizing these models for edge deployment and extending the framework to non-facial video forgeries.
Summary updated