Transferable Adversarial Facial Images for Privacy Protection

Minghui Li, Jiangxiong Wang, Hao Zhang, Ziqi Zhou, Shengshan Hu, Xiaobing Pei

Abstract

As deep face recognition (FR) systems become increasingly pervasive, the risk of unauthorized large-scale tracking grows. This paper introduces GIFT (Guidance-Independent Adversarial Facial Images with Transferability), a framework designed to generate natural-looking "cloaked" face images that effectively deceive black-box FR systems. By optimizing the entire facial latent space ($\mathcal{F}$ space) of a pre-trained StyleGAN2 model through Global Adversarial Latent Search (GALS) and ensuring identity preservation via Key Landmark Regularization (KLR), GIFT achieves a 25% improvement in protection success over state-of-the-art methods while maintaining high visual quality.

Introduction

Unauthorized tracking via deep face recognition (FR) systems is a significant privacy concern. While adversarial face images—perturbations that cause FR systems to misidentify individuals while appearing normal to humans—offer a potential solution, existing methods suffer from a "trilemma":

  1. Visual Quality: Noise-based methods often leave visible, distracting artifacts.
  2. Transferability: Attacks that work on one FR model often fail on others, particularly commercial "black-box" APIs.
  3. Usability: High-quality methods often require impractical user-provided guidance, such as reference images or text prompts.

The GIFT framework aims to bridge these gaps by creating a method that is natural-looking, highly transferable, and completely independent of user-provided guidance.

The GIFT Framework

GIFT operates by manipulating the latent space of a pre-trained generative model (StyleGAN2). Unlike previous methods that focus on local features like makeup, GIFT optimizes the entire facial latent space to find adversarial directions that subtly modify pose, expression, and face shape.

The GIFT framework pipeline: from source image to protected image, illustrating how cloaked images are uploaded to social media to deceive malicious FR systems.

The process involves three main components:

  • GAN Inversion into $\mathcal{F}$ Space: The source image is mapped into the $\mathcal{F}$ latent space, which provides an optimal balance between reconstruction fidelity and the ability to "edit" the image for adversarial purposes.
  • Global Adversarial Latent Search (GALS): Instead of local tweaks, GALS optimizes the entire latent code. This holistic approach allows the model to find adversarial features that generalize better to unknown FR systems.
  • Key Landmark Regularization (KLR): To ensure the person in the photo still looks like themselves to a human, KLR uses semantic segmentation maps to keep the overall face shape and characteristics consistent during the optimization process.

Experimental Results

The researchers evaluated GIFT using the CelebA-HQ and LADN datasets against four deep FR models (MobileFace, IR152, IRSE50, FaceNet) and three commercial APIs (Face++, Aliyun, and Tencent).

Comparison of Protection Success Rates (PSR) between Global Adversarial Latent Search (GALS) and Local Adversarial Latent Search (LALS) across various face recognition models.

Key findings include:

  • Superior Transferability: GIFT achieved an average Protection Success Rate (PSR) of 89.73% on deep FR models, a roughly 25% improvement over previous state-of-the-art (SOTA) methods like CLIP2Protect and AMT-GAN.
  • Commercial API Effectiveness: In tests against real-world black-box systems, GIFT improved protection by an average of 10%.
  • High Visual Quality: GIFT maintained superior visual quality, achieving a lower FID score (31.19) than competitors, meaning the generated images look more like real photos.
  • Global vs. Local Search: The holistic approach of GALS proved far more effective than "Local Adversarial Latent Search" (LALS). While LALS might only change makeup, GALS harmoniously refines all facial characteristics, making the adversarial "noise" much harder for black-box systems to ignore.

Conclusion

GIFT demonstrates that global latent space optimization is a powerful tool for facial privacy. By moving beyond local perturbations and utilizing the $\mathcal{F}$ space of generative models, it provides a robust and user-friendly defense against unauthorized face recognition. While the system currently relies on ensemble surrogate models, future work will explore making it even more robust against adversarial training defenses and extending the framework to video-based privacy protection.

Summary updated

BibTeX
@inproceedings{li2024transferable,
  author = {Minghui Li and Jiangxiong Wang and Hao Zhang and Ziqi Zhou and Shengshan Hu and Xiaobing Pei},
  title = {{Transferable Adversarial Facial Images for Privacy Protection}},
  booktitle = {ACM Multimedia},
  year = {2024},
  doi = {10.1145/3664647.3681344},
}

Loading related content...

Figure