Multiple Contexts and Frequencies Aggregation Network forDeepfake Detection

Zifeng Li, Wenzhong Tang, Shijun Gao, Shuai Wang, Yanxiang WangarXiv 2024 · DOI · arXiv

Abstract

Deepfake detection remains a critical challenge as generative models produce increasingly realistic forgeries. Standard neural network backbones often fail to capture subtle local artifacts and high-frequency inconsistencies. We present MkfaNet (Multiple Contexts and Frequencies Aggregation Network), a novel backbone specifically engineered for deepfake detection. By integrating a Multi-Kernel Aggregator (MKA) for multi-scale spatial context and a Multi-Frequency Aggregator (MFA) for adaptive frequency reweighting, MkfaNet achieves state-of-the-art performance across multiple benchmarks with high parameter efficiency.

Introduction

Most current deepfake detectors rely on "off-the-shelf" backbones like Xception or EfficientNet. While effective for general object recognition, these models are not optimized for the specific "tells" of a deepfake. Generative models (GANs, Diffusion Models) often leave behind microscopic traces: subtle inconsistencies in facial textures or abnormal energy spikes in high-frequency domains. Standard convolutional neural networks (CNNs) tend to act as low-pass filters, focusing on global structures while smoothing over the high-frequency details where forgery artifacts reside. The challenge lies in building a model that maintains global facial understanding while remaining hyper-sensitive to local, high-frequency noise introduced by AI manipulation.

Methods: The MkfaNet Architecture

MkfaNet is a four-stage hierarchical architecture designed to replace standard backbones in deepfake detection pipelines. It introduces two innovative modules to address spatial and frequency-domain limitations.

The hierarchical architecture of MkfaNet featuring four stages of MKA and MFA modules.

Multi-Kernel Aggregator (MKA)

Instead of standard convolutions, the MKA utilizes depth-wise separable convolutions with varying dilation rates (1, 2, and 3) in parallel. This allows the model to capture features at multiple scales simultaneously, which is crucial for identifying blending artifacts near facial organs like the eyes, nose, and mouth. A SiLU-based gating mechanism adaptively fuses these multi-scale features.

Multi-Frequency Aggregator (MFA)

The MFA module addresses the frequency gap by decomposing feature maps into DC (Low-Frequency) and HC (High-Frequency) components. By adaptively reweighting these components, the network can amplify high-frequency signals indicative of forgery without compromising the structural integrity of the image representation.

Experiments and Results

MkfaNet was evaluated across seven major benchmarks, including FaceForensics++ (FF++), CelebDF, and the DeepFake Detection Challenge (DFDC).

Performance and Efficiency

The "Small" variant, MkfaNet-S (20M parameters), achieved an average AUC of 94.85% in within-domain tests, outperforming larger models like Xception and EfficientNet-B4. In cross-domain evaluations—a key metric for real-world robustness—MkfaNet-S outperformed ConvNeXt-T by 1.13% in average AUC. Even the "Tiny" version (MkfaNet-T) with only 5.2M parameters outperformed ResNet-34 (22M parameters), demonstrating that domain-specific architectural design is more effective than raw scale.

Localization and Interpretability

Grad-CAM visualizations reveal that MkfaNet focuses more precisely on specific facial organs compared to standard models. While general-purpose backbones often look at broad facial regions, MkfaNet targets the "seams" and local textures where manipulation artifacts are most prevalent.

Grad-CAM visualizations comparing MkfaNet with standard backbones, showing superior localization of forgery artifacts.

Conclusion

MkfaNet addresses the "frequency blind spot" of traditional CNNs by explicitly aggregating multi-scale spatial contexts and frequency components. Its superior performance and parameter efficiency make it a robust backbone for next-generation deepfake detection. Future work will explore integrating temporal consistency to extend these spatial-frequency insights into the video domain.

Summary updated

BibTeX
@misc{li2024multiple,
  author = {Zifeng Li and Wenzhong Tang and Shijun Gao and Shuai Wang and Yanxiang Wang},
  title = {{Multiple Contexts and Frequencies Aggregation Network forDeepfake Detection}},
  year = {2024},
  eprint = {2408.01668},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2408.01668},
}
Figure