OPOM: Customized Invisible Cloak towards Face Privacy Protection

Yaoyao Zhong, Weihong Deng

Introduction

The rapid advancement of face recognition (FR) technology has brought significant convenience but also raised substantial privacy concerns. Unauthorized data harvesting from social media platforms allows for surreptitious surveillance. To counter this, researchers have developed adversarial "masks"—imperceptible digital noise that misleads AI models. However, existing solutions face a trade-off: image-specific masks (e.g., Fawkes, LowKey) are computationally expensive, while general universal masks often provide insufficient protection for specific individuals.

This paper introduces OPOM (One Person One Mask), a framework that generates a single, person-specific "invisible cloak." By creating one adversarial mask per identity, OPOM achieves a high-efficiency, high-protection balance, enabling real-time protection for all of a user's photos and videos.

Methodology

The OPOM framework aims to find a perturbation ($\Delta X$) that, when added to any image of a specific person $k$, pushes the resulting feature vector away from the "feature subspace" representing their identity in an FR model.

Figure 2: The OPOM framework pipeline, illustrating the iterative training process to generate a customized mask based on a feature subspace modeled from a few images.

The researchers explored three geometric representations for this identity subspace:

  1. Class Centers: Modeling identity as a single average point.
  2. Affine Hulls: Representing identity as any linear combination of the user's features.
  3. Convex Hulls: Modeling identity as the smallest convex set containing all training features.

OPOM-ConvexHull proved most effective, as the "region-based" approach provides more diverse gradient information during optimization, enhancing robustness against variations in pose or lighting. To improve "black-box" transferability, the authors integrated Momentum Boosting and DFANet, which uses dropout to simulate diverse network architectures during mask generation.

Experimental Evaluation

The method was evaluated on three benchmarks:

  • Privacy-Commons: 500 identities from MegaFace (10 training, 5 testing images per ID).
  • Privacy-Celebrities: 500 identities from a celebrity list (10 training, 10 testing images per ID).
  • Sherlock: A video dataset featuring characters from the TV series Sherlock for practical validation.

Key Results and Discussion

OPOM-ConvexHull consistently outperformed other variants and the baseline FI-UAP in both individual universality and model transferability.

Figure 1: Comparison of OPOM variants and FI-UAP across Privacy-Commons and Privacy-Celebrities datasets, showing the superior performance of OPOM-ConvexHull.

Performance Highlights

  • High Protection Success: On Privacy-Commons, OPOM-ConvexHull achieved a Top-1 protection success rate of approximately 86% against black-box models.
  • Efficiency: Once the initial mask is generated, it has a 0-second generation time for new images, making it uniquely suited for real-time video protection.
  • Commercial Viability: The masks successfully defeated major commercial APIs, including Amazon Rekognition, Microsoft Azure, Baidu Cloud, and Face++, significantly reducing similarity scores for the same individual.
  • Video Robustness: In the Sherlock case study, a single person-specific mask effectively prevented recognition across various frames despite changes in pose and expression.

Conclusion and Future Work

OPOM provides a practical middle ground for digital privacy. While highly effective, the authors note that extreme variance in pose or illumination can still degrade performance. Future research directions include addressing "mask leakage"—where malicious systems might learn to recognize masked faces—and transitioning these digital adversarial patterns into physical-world items like glasses or clothing.

Summary updated

Abstract

While convenient in daily life, face recognition technologies also raise privacy concerns for regular users on the social media since they could be used to analyze face images and videos, efficiently and surreptitiously without any security restrictions. In this paper, we investigate the face privacy protection from a technology standpoint based on a new type of customized cloak, which can be applied to all the images of a regular user, to prevent malicious face recognition systems from uncovering their identity. Specifically, we propose a new method, named one person one mask (OPOM), to generate person-specific (class-wise) universal masks by optimizing each training sample in the direction away from the feature subspace of the source identity. To make full use of the limited training images, we investigate several modeling methods, including affine hulls, class centers, and convex hulls, to obtain a better description of the feature subspace of source identities. The effectiveness of the proposed method is evaluated on both common and celebrity datasets against black-box face recognition models with different loss functions and network architectures. In addition, we discuss the advantages and potential problems of the proposed method. In particular, we conduct an application study on the privacy protection of a video dataset, Sherlock, to demonstrate the potential practical usage of the proposed method. Datasets and code are available at https://github.com/zhongyy/OPOM.
BibTeX
@article{zhong2023opom,
  author = {Yaoyao Zhong and Weihong Deng},
  title = {{OPOM: Customized Invisible Cloak towards Face Privacy Protection}},
  journal = {IEEE Trans. Pattern Anal. Mach. Intell.},
  year = {2023},
  url = {https://openreview.net/forum?id=euz_xV3K1hu},
}
Figure