Face Encryption via Frequency-Restricted Identity-Agnostic Attacks

Xin Dong, Rui Wang, Siyuan Liang, Aishan Liu, Lihua Jing

TL;DR

This paper introduces FRIA, a novel framework for encrypting face images to protect privacy from unauthorized facial recognition systems. FRIA achieves this by generating imperceptible perturbations that are identity-agnostic and restricted to low-frequency regions, ensuring both effective privacy protection and natural visual appearance.

What Problem Does This Paper Address?

In today's digital age, billions of people share images online, making their facial biometric data vulnerable to collection and misuse by malicious actors. Deep facial recognition systems can easily extract this sensitive information. While adversarial attacks have been proposed to "encrypt" faces by adding subtle perturbations, existing methods suffer from two major drawbacks: 1) Weak black-box scenario feasibility: they often require knowledge of the target recognition system or have poor transferability to unknown models. 2) Undesirable visual naturalness: the generated "encrypted" faces can look unnatural or altered, making users hesitant to share them. This paper aims to bridge this gap by developing a method that is effective against unknown systems and maintains the visual integrity of the original images.

Methods

The FRIA framework combines two key strategies:

  • Identity-Agnostic Attack: Instead of targeting specific identities, FRIA leverages the observation that average facial features across different recognition models are highly similar. By generating perturbations that make the encrypted face's features close to this "average feature" (derived from a reference set of diverse faces) and distant from the original face's features, the attack becomes transferable across various face recognition models without needing to know their specific identities. This is achieved using a loss function that encourages the encrypted face's features to move towards the average feature and away from the original face's features.
  • Low-Frequency Restricted Perturbation: To ensure visual naturalness, FRIA restricts perturbations to the high-frequency components of the image while keeping low-frequency components (which correspond to smoother regions and are more perceptible to the human visual system) less altered. This is accomplished by applying a mask in the frequency domain (using Discrete Cosine Transform - DCT) to isolate low-frequency information. The loss function is designed to minimize the difference between the original and encrypted images, with a focus on preserving the low-frequency content.

An adaptive weight adjustment strategy is employed during training to balance the competing objectives of attack effectiveness and image naturalness, preventing one from dominating the other.

Datasets

  • LFW (Labeled Faces in the Wild) and MegFace: Used for creating gallery and probe sets for evaluating black-box attack performance.
  • Celeb-A: Used to construct the reference set (Dref) for calculating the average features for the identity-agnostic attack.

No new benchmarks were introduced; existing datasets were utilized for evaluation.

Key Insights and Ideas

  • Average features as a universal target: The surprising similarity of average facial features across diverse face recognition models provides a robust, identity-agnostic target for adversarial attacks, significantly improving transferability.
  • Frequency domain for naturalness: Restricting perturbations to high-frequency components in the frequency domain is a more effective way to maintain visual naturalness than manipulating the RGB domain directly. This is because human vision is more sensitive to changes in smooth regions (low frequencies).
  • Balancing act: The inherent conflict between maximizing attack success and preserving naturalness requires careful balancing, which the proposed adaptive weight adjustment strategy effectively addresses.

Key Results

  • High Black-Box Attack Success: FRIA achieved an Encrypted Success Rank-1 (ESR-1) of over 95% on the majority of tested face recognition models in black-box scenarios, outperforming previous state-of-the-art methods by up to 11.4%. ESR-5 also showed significant improvements.
  • Superior Visual Naturalness: FRIA demonstrated the best performance across multiple naturalness metrics, including PSNR, SSIM, and human perception studies, with response scores significantly higher than other methods. Qualitative results (Figure 4) visually confirm the natural appearance of FRIA-generated encrypted faces.
  • Real-world API Efficacy: FRIA successfully confused a commercial face recognition API (Face++), significantly reducing similarity scores and increasing the average ranking position of true identities, demonstrating practical applicability.
  • Robustness to Defenses: FRIA showed strong performance even after the application of common defense mechanisms like BitRed and JPEG compression, indicating its resilience.

Limitations and Future Work

  • Limitations: While FRIA significantly improves naturalness, the paper acknowledges that the "average feature" approach might not be universally optimal for all possible future face recognition architectures. The current method focuses on specific frequency restrictions, and further exploration of adaptive frequency manipulation could yield even better results.
  • Future Work: The authors suggest designing strategies for adaptively learning perturbation magnitudes at specific frequency domains. They also express interest in implementing FRIA as a pre-processing tool for social media websites to enhance user privacy proactively.

Summary updated

Abstract

Billions of people are sharing their daily live images on social media everyday. However, malicious collectors use deep face recognition systems to easily steal their biometric information (e.g., faces) from these images. Some studies are being conducted to generate encrypted face photos using adversarial attacks by introducing imperceptible perturbations to reduce face information leakage. However, existing studies need stronger black-box scenario feasibility and more natural visual appearances, which challenge the feasibility of privacy protection. To address these problems, we propose a frequency-restricted identity-agnostic (FRIA) framework to encrypt face images from unauthorized face recognition without access to personal information. As for the weak black-box scenario feasibility, we obverse that representations of the average feature in multiple face recognition models are similar, thus we propose to utilize the average feature via the crawled dataset from the Internet as the target to guide the generation, which is also agnostic to identities of unknown face recognition systems; in nature, the low-frequency perturbations are more visually perceptible by the human vision system. Inspired by this, we restrict the perturbation in the low-frequency facial regions by discrete cosine transform to achieve the visual naturalness guarantee. Extensive experiments on several face recognition models demonstrate that our FRIA outperforms other state-of-the-art methods in generating more natural encrypted faces while attaining high black-box attack success rates of 96%. In addition, we validate the efficacy of FRIA using real-world black-box commercial API, which reveals the potential of FRIA in practice. Our codes can be found in https://github.com/XinDong10/FRIA.
BibTeX
@inproceedings{dong2023face,
  author = {Xin Dong and Rui Wang and Siyuan Liang and Aishan Liu and Lihua Jing},
  title = {{Face Encryption via Frequency-Restricted Identity-Agnostic Attacks}},
  booktitle = {ACM Multimedia},
  year = {2023},
  doi = {10.1145/3581783.3612233},
}
Figure