Which Face and Whose Identity? Solving the Dual Challenge of Deepfake Proactive Forensics in Multi-Face Scenarios

Lei Zhang, Zhiqing Guo, Dan Ma, Gaobo YangarXiv 2026 · arXiv · OpenReview

TL;DR

This paper introduces DAWF, a novel framework for proactive deepfake forensics in complex multi-person scenarios. DAWF uniquely addresses the challenges of identifying which faces are forged and whose identities are tampered with, even in images with multiple individuals, by embedding identity-specific watermarks and using a selective loss function to focus on manipulated regions.

What Problem Does This Paper Address?

The proliferation of deepfake technology has extended beyond single-person manipulations to sophisticated forgeries involving multiple individuals in group photos or meeting recordings. Existing deepfake detection methods, particularly proactive forensics (which embed watermarks before potential manipulation), typically operate under a "single-face" assumption. This limitation makes them ill-equipped to handle the complexities of multi-person scenarios, where low-resolution individual faces, overlapping identities, and cross-face interference can severely degrade detection, localization, and source tracing capabilities. The paper aims to bridge this gap by developing a framework that can reliably perform these forensic tasks in these challenging multi-face environments.

Methods

The core of the proposed solution is the Deep Attributable Watermarking Framework (DAWF). DAWF comprises two main components:

  • MFEncoder: This component first detects all facial regions in a cover image. It then uses a novel steganographic kernel with differentiable bilinear interpolation to create an "isolated identity attribution space." This allows for the embedding of unique, identity-specific watermarks into each detected face. The framework processes these faces in parallel, enabling efficient embedding even in high-resolution images.
  • MFDecoder (MFTracer & MFLocalizer): After the image is potentially subjected to various distortions (including deepfake manipulations), the MFDecoder attempts to recover the embedded watermarks.
  • The MFTracer is designed for robust watermark recovery, aiming to reconstruct the original watermark even under significant degradation, thus enabling source tracing.
  • The MFLocalizer is designed to be semi-fragile. It aims to recover watermarks accurately under benign distortions but is explicitly trained to produce severely discordant outputs when malicious deepfake manipulations are present.
  • A key innovation is the selective regional supervision loss. This loss function actively filters out authentic facial regions and guides the MFLocalizer to focus exclusively on the forged areas, thereby suppressing cross-face interference and improving localization accuracy.

Datasets

The primary dataset used for training and evaluation is WIDERFace [31]. This dataset is noted for being a widely used and challenging benchmark for multi-face detection, featuring diverse scene backgrounds and face distributions. The authors filtered this dataset to include facial instances with bounding boxes larger than 20x20 pixels, as smaller faces are considered less suitable for high-fidelity face swapping and forensic evaluation. They also conducted cross-dataset generalizability tests on COCO2017 [39], OpenForensics [2], and CelebA-HQ [40].

Key Insights and Ideas

  • The "Which + Who" Challenge: The paper frames the problem as a dual challenge: not only identifying which faces are manipulated but also determining whose identity was tampered with. This is a significant step beyond traditional single-face detection.
  • Isolated Identity Attribution Space: The concept of creating a dedicated space for each face's watermark is crucial. It allows multiple independent tracing signals to coexist within a single image without interfering with each other.
  • Selective Regional Supervision: This is a critical component for handling multi-face scenarios. By explicitly guiding the model to focus only on manipulated regions and ignore authentic ones, it effectively tackles the problem of cross-face interference, which is a major hurdle in existing methods.
  • Proactive Forensics for Multi-Face: The paper highlights that existing proactive forensics methods, while effective for single faces, fail to scale to multi-face scenarios due to resolution limitations and interference. DAWF directly addresses this by adapting proactive forensics for these complex environments.

Key Results

  • High Accuracy: DAWF achieves impressive performance on challenging multi-face datasets, reporting an AUC of 0.91, an F1-score of 0.87, and a BER of 0.54%.
  • Robustness: The framework demonstrates strong robustness against various common distortions (e.g., JPEG compression, Gaussian blur) and malicious deepfake attacks (e.g., Simswap, Ghost, Mobilefaceswap, CSCS).
  • Visual Quality: The encoded images maintain high visual fidelity, with PSNR and SSIM scores comparable to or better than existing methods, even at high resolutions.
  • Efficient Multi-Identity Tracing: DAWF can embed a dynamic payload (15xN bits for N faces) and naturally scales with the number of faces, offering flexibility for complex scenarios.
  • Cross-Dataset Generalizability: The model shows exceptional performance consistency across different datasets (COCO2017, OpenForensics, CelebA-HQ), indicating its broad applicability.

Limitations and Future Work

  • Payload Capacity vs. Resolution Trade-off: The paper acknowledges an inherent trade-off between payload capacity and local resolution. While DAWF can support larger payloads with higher resolutions, this comes at the cost of increased computational complexity and potentially reduced payload density.
  • Future Work: The authors suggest future work should focus on improving payload capacity for low-resolution areas while maintaining robustness and imperceptibility. Additionally, enhancing generalization against more complex distortions and unknown forgery technologies is a key direction.

Summary updated

Abstract

Unlike single-face forgeries, deepfakes in complex multi-person interaction scenarios (such as group photos and multi-person meetings) more closely reflect real-world threats. Although existing proactive forensics solutions demonstrate good performance, they heavily rely on a "single-face" setting, making it difficult to effectively address the problems of deepfake localization and source tracing in complex multi-person environments. To address this challenge, we propose the Deep Attributable Watermarking Framework (DAWF). This framework adopts a novel multi-face encoder-decoder architecture that bypasses the cumbersome offline pre-processing steps of traditional forensics, facilitating efficient in-network parallel watermark embedding and cross-face collaborative processing. Crucially, we propose a selective regional supervision loss. This innovative mechanism guides the decoder to focus exclusively on the facial regions tampered with by deepfakes. Leveraging this mechanism alongside the embedded identity payloads, DAWF realizes the "which + who" goal, answering the dual questions of which facial region was forged and who was forged. Extensive experiments on challenging multi-face datasets show that DAWF achieves excellent deepfake localization and traceability in complex multi-person scenes.
BibTeX
@misc{zhang2026which,
  author = {Lei Zhang and Zhiqing Guo and Dan Ma and Gaobo Yang},
  title = {{Which Face and Whose Identity? Solving the Dual Challenge of Deepfake Proactive Forensics in Multi-Face Scenarios}},
  year = {2026},
  eprint = {2604.26342},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2604.26342},
}
Figure