Abstract
As Generative Adversarial Networks (GANs) produce increasingly realistic human faces, the need for robust detection methods grows. This paper introduces FakeSpotter, a novel detection framework that identifies AI-synthesized faces by monitoring the internal neuron activation patterns of pre-trained face recognition models. By treating layer-by-layer activations as features for a simple classifier, FakeSpotter achieves high accuracy and remarkable robustness against common image perturbations like compression and blurring, outperforming existing pixel-based detection methods.
Introduction
Generative Adversarial Networks (GANs) have reached a point where they can synthesize human faces that are virtually indistinguishable from real ones to the naked eye. This technology powers various manipulations, ranging from "entire synthesis" of non-existent people to "DeepFakes" involving identity swapping.
A core challenge in the field is that existing fake-face detectors are often brittle. Many rely on specific pixel-level artifacts or "fingerprints" left by specific GAN architectures. When these images are shared online, they undergo transformations—resizing, JPEG compression, or blurring—that often "wash away" these subtle artifacts, causing detectors to fail. Furthermore, as GANs evolve, these specific artifacts disappear. FakeSpotter seeks a more universal and robust baseline by examining how deep networks perceive faces rather than focusing solely on raw pixels.
Methodology
The authors propose that the internal state of a deep neural network contains more information about an image's authenticity than the final output alone. The intuition is that GAN-generated faces, while visually convincing, trigger different layer-by-layer activation patterns in a model trained on millions of real faces.
The Detection Pipeline
- The "Sensor" Model: A pre-trained, high-performing Face Recognition (FR) system (specifically VGG-Face with a ResNet-50 backbone) is used as a feature extractor.
- Mean Neuron Coverage (MNC): Instead of raw activation values, a threshold $\xi_l$ is defined for each layer $l$ based on the average activation values observed during training.
- Feature Extraction: For any input image, the system calculates the "neuron coverage"—a count of how many neurons in each layer are activated above the threshold. This transforms a high-dimensional image into a relatively small feature vector where each element represents a layer's activation density.
- Binary Classification: These feature vectors are fed into a simple, shallow neural network (five fully-connected layers) trained to distinguish between "Real" and "Fake" patterns.
Experiments and Results
The researchers evaluated FakeSpotter against four distinct categories of face synthesis: entire synthesis (PGGAN, StyleGAN2), attribute editing (StarGAN, STGAN), expression manipulation (StyleGAN, STGAN), and identity swapping (FaceForensics++, DFDC, Celeb-DF).
Performance Metrics
- High Accuracy: FakeSpotter achieved an average detection accuracy of over 90% across the four types of fakes. It specifically reached 98.6% on PGGAN and 91.8% on the state-of-the-art StyleGAN2.
- Superior Robustness: In tests against four common perturbations (noise, blur, compression, and resizing), FakeSpotter’s performance (measured by AUC) dropped by less than 3.77% on average. This significantly outperformed prior work like AutoGAN, which is much more sensitive to image quality.
- Benchmark Performance: On the challenging Celeb-DF(v2) dataset, FakeSpotter reached an AUC of 66.8%, outperforming 13 other existing detection methods at the time of publication.
Discussion and Conclusion
The most significant insight of this work is that internal neuron behaviors serve as a powerful forensic tool. While a GAN might fool the final classification layer, the subtle features captured in intermediate layers often reveal the synthesis. Furthermore, because the method relies on the aggregate count of activated neurons per layer rather than specific pixel values, it is inherently more resistant to local noise.
Limitations
Despite its success, FakeSpotter has limitations. Its performance on the DFDC dataset was lower, likely because those fakes often involve both face and voice swaps, whereas FakeSpotter is purely visual. Additionally, while robust to random noise, the system could potentially be targeted by "adversarial fakes" designed to mimic real neuron activation patterns. Future work may focus on fake localization—pinpointing exactly which parts of a face have been manipulated.
Summary updated