Enhancing Facial Privacy Protection via Weakening Diffusion Purification

Ali Salar, Qing Liu, Yingli Tian, Guoying Zhao

Introduction

The proliferation of personal images on social media has fueled the growth of unauthorized automatic face recognition (AFR) systems. These systems enable mass surveillance, posing significant privacy risks to individuals. To counter this, researchers have developed "targeted de-identification" techniques. These methods aim to modify images such that they remain recognizable to humans as the original person but are misidentified by AI systems as a different "target" identity.

The Challenge: The Diffusion Purification Paradox

Recent advancements in generative AI have led to the use of Diffusion Models for creating adversarial faces. However, these models face a fundamental hurdle known as diffusion purification. Because diffusion models are designed to remove noise and reconstruct clean images, the reverse diffusion process often inadvertently "cleans" the high-frequency adversarial perturbations required to fool AFR systems. This leads to a low Protection Success Rate (PSR). Additionally, many existing methods cause noticeable visual distortions by warping the underlying physical structure of the face.

Proposed Methodology

This paper introduces a two-stage framework built on a Latent Diffusion Model (LDM) to overcome these limitations.

1. Weakening Purification with Unconditional Embeddings

Instead of relying on standard text prompts, the method optimizes per-timestamp unconditional embeddings (null-text guidance). These learned embeddings act as a structural anchor, providing the model with the capacity to retain fine textures and preventing it from over-purifying the adversarial noise during the denoising process.

The proposed method (Ours) generates high-quality adversarial faces that maintain natural appearance while achieving high protection scores (73.88).

2. Adversarial Latent Code and Self-Attention Guidance

The framework directly optimizes the adversarial latent code ($z_{adv}$) to minimize feature distance to a target identity while maximizing distance from the original. To ensure the face's geometric structure is preserved, the authors align the self-attention maps of the adversarial generation with those of the original image. Unlike cross-attention, self-attention maps capture the inherent shape and spatial details of the face, preventing identity morphing.

Experimental Evaluation

The method was evaluated using the CelebA-HQ and LADN datasets, with AI-generated faces used as "fake" targets to avoid ethical concerns regarding impersonating real individuals.

Performance and Quality

The proposed approach achieved a state-of-the-art average PSR of 79.17% across multiple black-box FR models (including IRSE50 and Facenet), significantly outperforming previous diffusion-based methods like DiffProtect (51.05%). Furthermore, it achieved a Fréchet Inception Distance (FID) of 15.32, indicating superior visual naturalness compared to baselines.

Baseline methods such as TIP-IM often suffer from lower protection success rates (46.06) and visible adversarial artifacts.

Robustness and Real-World Efficacy

The adversarial faces remained effective even against image-smoothing defenses like Gaussian blurring. Testing against commercial APIs from Face++ and Tencent further confirmed the method's real-world efficacy, where it consistently outperformed baseline methods in fooling commercial-grade recognition systems.

Conclusion

By addressing the diffusion purification effect and utilizing self-attention for structural preservation, this work sets a new standard for facial privacy protection. While the current generation time of 15 seconds per image is a bottleneck compared to GAN-based methods, future work will focus on optimizing the attack strategy within the UNet's semantic space to enable faster, real-world deployment.

Summary updated

BibTeX
@inproceedings{salar2025enhancing,
  author = {Ali Salar and Qing Liu and Yingli Tian and Guoying Zhao},
  title = {{Enhancing Facial Privacy Protection via Weakening Diffusion Purification}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2025},
  doi = {10.1109/cvpr52734.2025.00771},
}
Figure