Face De-identification: State-of-the-art Methods and Comparative Studies

Jingyi Cao, Xiangyi Chen, Bo Liu, Ming Ding, Rong Xie, Li Song, Zhu Li, Wenjun Zhang

TL;DR

This paper provides a comprehensive survey and comparative analysis of face de-identification techniques, categorizing them into pixel, representation, and semantic levels. By evaluating both traditional approaches and modern deep generative models (GANs and diffusion models), the authors highlight how different frameworks navigate the critical trade-off between robust identity privacy and image utility preservation.

What Problem Does This Paper Address?

The rapid proliferation of deep learning-based facial recognition systems has created significant privacy risks. Because facial biometrics are unique and irrevocable, unauthorized cross-referencing across databases can expose sensitive personal data, violating regulations like GDPR.

To combat this, face de-identification (anonymization) techniques alter facial images to prevent automated recognition or human identification. However, the core challenge lies in balancing a strict trade-off: maximizing privacy protection while preserving the image's utility for non-identity tasks (such as facial expression recognition, action detection, or medical diagnostics). Existing literature lacks a unified, up-to-date survey that systematically categorizes and quantitatively compares modern deep learning approaches against these competing criteria.

Methods

The authors propose a structured, level-based algorithmic taxonomy to classify face de-identification methods:

  1. Pixel-Level: Atomic operations performed directly on raw pixels. This includes traditional methods (blurring, pixelation, masking), the k-Same family (averaging pixel data across clusters to achieve k-anonymity), and generative face inpainting (completely masking the face and using conditional GANs to reconstruct a realistic, synthetic face).
  2. Representation-Level: Anonymization performed within a downsampled latent space. This involves either manipulating latent vectors within generative models (GANs, VAEs, or diffusion models) or applying adversarial perturbations—adding imperceptible noise designed to fool face recognition networks while leaving the image visually unchanged for humans.
  3. Semantic-Level: Extracting and obfuscating specific semantic features. This is split into attribute-level methods (modifying non-identity traits like age or gender) and identity-level methods, which explicitly disentangle identity embeddings from attribute embeddings, altering only the identity components to maximize visual fidelity and utility.

The survey also details how these methods integrate formal privacy frameworks, such as k-anonymity and Differential Privacy (DP), to provide mathematical privacy guarantees.

Fig. 1. Taxonomy of face de-identification methods. We group existing methods according to process level and then discuss de-identification in videos. In addition, several de-identification methods are combined with privacy theory, like k-anonymity and differential privacy, to provide privacy guarantees.

Datasets

While the survey reviews literature spanning numerous datasets (including WIDER-Face, CASIA, LFW, FFHQ, VGGFace2, and VoxCeleb), the authors conduct their own standardized experimental comparison using:

  • CelebA-HQ: A high-quality dataset of 30,000 celebrity facial images cropped and normalized to a $256 \times 256$ resolution.

No new datasets were introduced in this paper.

Key Insights and Ideas

  • Disentanglement is Key: Semantic identity-level methods consistently achieve the best balance between privacy and utility. By cleanly separating identity from attributes (like expression and pose), they allow targeted modifications without degrading the image's secondary uses.
  • The Fragility of Adversarial Noise: Although adversarial perturbation methods (like Fawkes) preserve near-perfect image quality for human viewers, they are highly brittle. They often fail to generalize against black-box face recognition systems that they weren't explicitly trained to fool.
  • Generative Trade-offs: Pixel-level face generation (inpainting) offers the highest level of privacy because the original face is entirely destroyed, but it completely discards appearance similarity. Conversely, representation-level editing preserves structure well but lacks flexibility once the model is trained.
  • Video Anonymization Bottlenecks: Moving from still images to video introduces severe temporal consistency challenges. Without strict spatio-temporal constraints or dense motion flow tracking, de-identified videos suffer from jarring artifacts like flickering and identity warping.

Key Results

The authors conducted quantitative evaluations on CelebA-HQ using FaceNet variants to measure the Successful De-identification Rate (SDR) and various utility metrics (SSIM, PSNR, LPIPS):

  • High-Privacy Performers: Representation and semantic generative models like Gu et al., IdentityDP ($\epsilon=0.57$), and Zhu et al. achieved excellent privacy protection, yielding SDRs above 90% across multiple face recognition models.
  • Utility Preservation: Adversarial perturbation via Fawkes maintained an outstanding structural similarity (SSIM of 0.994) and minimal perceptual distortion (LPIPS of 0.005). However, in the authors' black-box testing environment, its SDR dropped to 0%, highlighting its real-world limitations.
  • Balanced Approaches: IdentityDP and Zhu et al. struck the most reliable balance, maintaining high de-identification rates (>90%) while retaining solid image quality (SSIM scores around 0.86–0.87).

Limitations and Future Work

  • Lack of Standardized Metrics: There is currently no universally accepted evaluation framework or benchmark dataset tailored specifically for face de-identification, making cross-paper comparisons difficult.
  • Generalization and Bias: Most deep learning frameworks suffer from dataset bias and struggle to maintain performance when deployed on diverse, real-world, non-frontal facial images.
  • Reversibility: The majority of modern methods are irreversible. The authors note that developing secure, password-controlled reversible frameworks (allowing authorized decryption for tasks like law enforcement) is a crucial future direction.
  • Multi-modal Expansion: As AI systems advance, facial data is increasingly fused with other biometrics. Future research must explore multi-modal privacy systems that simultaneously anonymize faces, voices, and gaits.

Summary updated

Abstract

The widespread use of image acquisition technologies, along with advances in facial recognition, has raised serious privacy concerns. Face de-identification usually refers to the process of concealing or replacing personal identifiers, which is regarded as an effective means to protect the privacy of facial images. A significant number of methods for face de-identification have been proposed in recent years. In this survey, we provide a comprehensive review of state-of-the-art face de-identification methods, categorized into three levels: pixel-level, representation-level, and semantic-level techniques. We systematically evaluate these methods based on two key criteria, the effectiveness of privacy protection and preservation of image utility, highlighting their advantages and limitations. Our analysis includes qualitative and quantitative comparisons of the main algorithms, demonstrating that deep learning-based approaches, particularly those using Generative Adversarial Networks (GANs) and diffusion models, have achieved significant advancements in balancing privacy and utility. Experimental results reveal that while recent methods demonstrate strong privacy protection, trade-offs remain in visual fidelity and computational complexity. This survey not only summarizes the current landscape but also identifies key challenges and future research directions in face de-identification.
BibTeX
@misc{cao2024face,
  author = {Jingyi Cao and Xiangyi Chen and Bo Liu and Ming Ding and Rong Xie and Li Song and Zhu Li and Wenjun Zhang},
  title = {{Face De-identification: State-of-the-art Methods and Comparative Studies}},
  year = {2024},
  eprint = {2411.09863},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2411.09863},
}
Figure