Detection of Digital Manipulation in Facial Images (Student Abstract)

Aman Mehra, Akshay Agarwal, Mayank Vatsa, Richa Singh

Abstract

As DeepFake technology advances toward photorealism, the need for robust detection systems becomes critical. This paper introduces a novel detection pipeline that leverages Eulerian motion magnification to expose subtle temporal inconsistencies in forged videos, particularly those subject to heavy compression. Additionally, we address the prevalent issue of demographic bias in facial forgery detection. By introducing the Indian Forgery Dataset and conducting a systematic study on training data composition, we demonstrate that achieving a "critical mass" of diversity—specifically a 30% representation of under-represented groups—is sufficient to significantly mitigate the generalization gap in detection accuracy.

Introduction

The proliferation of DeepFakes poses significant societal risks, from misinformation to personal harassment. Current detection methods often fail in real-world scenarios due to two primary factors: high video compression, which erases fine spatial artifacts, and demographic bias, resulting from training sets dominated by Caucasian subjects. This research proposes a motion-centric detection approach and evaluates the impact of ethnic diversity on model fairness.

Methodology: Motion Magnification and 3D-SE-ResNet

Traditional detectors often rely on spatial artifacts like skin texture or eye reflections. However, video compression (e.g., H.264) typically smooths these details. Our approach shifts focus to temporal patterns using Eulerian Motion Magnification. This technique amplifies pixel-level changes over time, making the jittery "stitching" artifacts of DeepFakes visible to neural networks.

The core architecture is a modified 3D-SE-ResNet. By incorporating dense skip connections between residual blocks, the model efficiently captures multi-scale features. The network processes a concatenated input of the magnified video clip and a "difference map" (the delta between magnified and original frames), allowing it to learn both temporal context and localized manipulation cues.

Addressing Demographic Bias: The Indian Forgery Dataset

To combat the "generalization gap" where models underperform on non-Caucasian faces, we introduced the Indian Forgery Dataset. This dataset contains 200 pristine and 248 forged videos (created via FSGAN) of Indian subjects, featuring 149 unique identities.

We conducted a bias study by varying the "Merging Fraction" ($\alpha$), representing the proportion of Indian vs. Caucasian samples in the training set. We measured the performance gap between the majority and minority groups to determine the optimal data balance for fairness.

Table 1: Generalization gap metrics across different merging fractions ($\alpha$) for various architectures, showing the impact of dataset diversity on model fairness.

Our findings suggest a "30% Rule" for Fairness: once the under-represented group reaches approximately 30% of the training data ($\alpha = 0.3$), the performance gap narrows significantly. This indicates that perfectly balanced datasets are not strictly necessary to achieve equitable detection performance, provided a critical threshold of diversity is met.

Experimental Results

The proposed method was evaluated against standard benchmarks, including FaceForensics++ and Celeb-DF. Our 3D-SE-ResNet consistently outperformed established models like MesoNet and XceptionNet, particularly on high-compression videos where spatial detectors typically degrade.

Figure 1: Performance comparison of the proposed method against MesoNet and XceptionNet across multiple DeepFake datasets, highlighting superior accuracy on high-quality and compressed benchmarks.

Ablation studies confirmed that the integration of dense connections and motion magnification provided incremental gains in accuracy across all tested datasets (DF, F2F, FS, NT, FSh), validating the multi-component approach.

Conclusion and Future Work

By prioritizing temporal motion over spatial appearance, our method offers a more resilient solution to compressed DeepFake detection. Furthermore, our work highlights the importance of ethnic diversity in training data, providing a practical guideline for building fairer AI systems. Future research will expand to a wider range of ethnicities and evaluate the system's performance against emerging generative techniques like StyleGAN-based swaps.

Summary updated

Abstract

Advances in deep learning have enabled the creation of photo-realistic DeepFakes by switching the identity or expression of individuals. Such technology in the wrong hands can seed chaos through blackmail, extortion, and forging false statements of influential individuals. This work proposes a novel approach to detect forged videos by magnifying their temporal inconsistencies. A study is also conducted to understand role of ethnicity bias due to skewed datasets on deepfake detection. A new dataset comprising forged videos of Indian ethnicity individuals is presented to facilitate this study.
BibTeX
@inproceedings{mehra2021detection,
  author = {Aman Mehra and Akshay Agarwal and Mayank Vatsa and Richa Singh},
  title = {{Detection of Digital Manipulation in Facial Images (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i18.17919},
}
Figure