TL;DR
This paper introduces DAWF, a novel framework for proactive deepfake forensics in complex multi-person scenarios. DAWF uniquely addresses the challenges of identifying which faces are forged and whose identities are tampered with, even in images with multiple individuals, by embedding identity-specific watermarks and using a selective loss function to focus on manipulated regions.
What Problem Does This Paper Address?
The proliferation of deepfake technology has extended beyond single-person manipulations to sophisticated forgeries involving multiple individuals in group photos or meeting recordings. Existing deepfake detection methods, particularly proactive forensics (which embed watermarks before potential manipulation), typically operate under a "single-face" assumption. This limitation makes them ill-equipped to handle the complexities of multi-person scenarios, where low-resolution individual faces, overlapping identities, and cross-face interference can severely degrade detection, localization, and source tracing capabilities. The paper aims to bridge this gap by developing a framework that can reliably perform these forensic tasks in these challenging multi-face environments.
Methods
The core of the proposed solution is the Deep Attributable Watermarking Framework (DAWF). DAWF comprises two main components:
- MFEncoder: This component first detects all facial regions in a cover image. It then uses a novel steganographic kernel with differentiable bilinear interpolation to create an "isolated identity attribution space." This allows for the embedding of unique, identity-specific watermarks into each detected face. The framework processes these faces in parallel, enabling efficient embedding even in high-resolution images.
- MFDecoder (MFTracer & MFLocalizer): After the image is potentially subjected to various distortions (including deepfake manipulations), the MFDecoder attempts to recover the embedded watermarks.
- The MFTracer is designed for robust watermark recovery, aiming to reconstruct the original watermark even under significant degradation, thus enabling source tracing.
- The MFLocalizer is designed to be semi-fragile. It aims to recover watermarks accurately under benign distortions but is explicitly trained to produce severely discordant outputs when malicious deepfake manipulations are present.
- A key innovation is the selective regional supervision loss. This loss function actively filters out authentic facial regions and guides the MFLocalizer to focus exclusively on the forged areas, thereby suppressing cross-face interference and improving localization accuracy.
Datasets
The primary dataset used for training and evaluation is WIDERFace [31]. This dataset is noted for being a widely used and challenging benchmark for multi-face detection, featuring diverse scene backgrounds and face distributions. The authors filtered this dataset to include facial instances with bounding boxes larger than 20x20 pixels, as smaller faces are considered less suitable for high-fidelity face swapping and forensic evaluation. They also conducted cross-dataset generalizability tests on COCO2017 [39], OpenForensics [2], and CelebA-HQ [40].
Key Insights and Ideas
- The "Which + Who" Challenge: The paper frames the problem as a dual challenge: not only identifying which faces are manipulated but also determining whose identity was tampered with. This is a significant step beyond traditional single-face detection.
- Isolated Identity Attribution Space: The concept of creating a dedicated space for each face's watermark is crucial. It allows multiple independent tracing signals to coexist within a single image without interfering with each other.
- Selective Regional Supervision: This is a critical component for handling multi-face scenarios. By explicitly guiding the model to focus only on manipulated regions and ignore authentic ones, it effectively tackles the problem of cross-face interference, which is a major hurdle in existing methods.
- Proactive Forensics for Multi-Face: The paper highlights that existing proactive forensics methods, while effective for single faces, fail to scale to multi-face scenarios due to resolution limitations and interference. DAWF directly addresses this by adapting proactive forensics for these complex environments.
Key Results
- High Accuracy: DAWF achieves impressive performance on challenging multi-face datasets, reporting an AUC of 0.91, an F1-score of 0.87, and a BER of 0.54%.
- Robustness: The framework demonstrates strong robustness against various common distortions (e.g., JPEG compression, Gaussian blur) and malicious deepfake attacks (e.g., Simswap, Ghost, Mobilefaceswap, CSCS).
- Visual Quality: The encoded images maintain high visual fidelity, with PSNR and SSIM scores comparable to or better than existing methods, even at high resolutions.
- Efficient Multi-Identity Tracing: DAWF can embed a dynamic payload (15xN bits for N faces) and naturally scales with the number of faces, offering flexibility for complex scenarios.
- Cross-Dataset Generalizability: The model shows exceptional performance consistency across different datasets (COCO2017, OpenForensics, CelebA-HQ), indicating its broad applicability.
Limitations and Future Work
- Payload Capacity vs. Resolution Trade-off: The paper acknowledges an inherent trade-off between payload capacity and local resolution. While DAWF can support larger payloads with higher resolutions, this comes at the cost of increased computational complexity and potentially reduced payload density.
- Future Work: The authors suggest future work should focus on improving payload capacity for low-resolution areas while maintaining robustness and imperceptibility. Additionally, enhancing generalization against more complex distortions and unknown forgery technologies is a key direction.
Summary updated