My Face My Choice: Privacy Enhancing Deepfakes for Social Media Anonymization

Umur A. Çiftçi, Gokturk Yuksek, İlke DemirWACV 2023

TL;DR

This paper introduces "My Face My Choice" (MFMC), a novel system that uses deepfakes to anonymize faces in social media photos based on user-defined access rights. By replacing faces with quantitatively dissimilar deepfakes, MFMC significantly reduces the accuracy of state-of-the-art face recognition systems, offering a practical solution for enhanced privacy in online photo sharing.

What Problem Does This Paper Address?

The proliferation of face recognition technology, while offering benefits, has raised significant privacy concerns. In social media, users often appear in numerous photos, and their facial data can be collected and used without explicit consent. Existing privacy-preserving methods like blurring or masking often alter the image in ways that are visually apparent and can be overcome by advanced recognition systems. Furthermore, current social media platforms handle privacy at the image level (who can see the photo), not at the face level (who can see my face in the photo). This leaves users with limited control over their digital identity and facial representation online, especially when their faces are captured in photos they didn't upload or approve.

Methods

MFMC tackles this problem by generating deepfakes of faces that are quantitatively dissimilar to the original but preserve general attributes like age and gender, as well as facial expressions and head poses. The core of the system involves:

  • Deepfake Generation: The paper evaluates several deepfake generation techniques, ultimately favoring SimSwap for its ability to transfer identity while preserving facial attributes and avoiding visible artifacts.
  • Target Face Selection: A key innovation is the metric for selecting target faces. Instead of simply finding the closest match, MFMC prioritizes finding faces that are maximally dissimilar in the embedding space, with options to consider age and gender similarity. This ensures the deepfake is distinct enough to fool recognition systems.
  • Access Models: MFMC proposes three access models ("Disclosure by Proxy," "Disclosure by Explicit Authorization," and "Access Rule Based Disclosure") that define how faces are anonymized based on user-defined social graphs and permissions. These models range from automatically faking all non-owner faces to requiring explicit tagging for real face display.
  • Face Embeddings: The system uses ArcFace to extract 512-dimensional face embeddings. These embeddings are crucial for measuring similarity and dissimilarity between faces.

Datasets

  • Real Dataset: The "party" subset of the Facial Descriptors Dataset [61] was used for real-world social interaction images.
  • Synthetic Datasets:
    • StyleGAN [29] generated dataset: Contains 10,000 deepfakes with random distributions of age, gender, and skin tones.
  • Generated.photos [27] dataset: Contains 10,000 deepfakes with equal distributions across age, gender, and skin tones.

Key Insights and Ideas

  • Deepfakes for Privacy: The paper demonstrates a responsible and beneficial application of deepfake technology for enhancing user privacy, contrasting with more dystopian uses.
  • Dissimilarity as a Privacy Metric: The core insight is that maximizing dissimilarity in the face embedding space is crucial for fooling face recognition systems, rather than just finding a visually similar replacement.
  • Face-Level Control: MFMC shifts privacy control from image-level to face-level, empowering users to decide who sees their face in any given photo.
  • Contextual Integrity: The system aims to preserve contextual integrity by maintaining attributes like age, gender, expression, and pose, making the deepfakes more believable within the image.

Key Results

  • Face Recognition Accuracy Reduction: MFMC reduces the average accuracy of seven state-of-the-art face recognition systems by 61%. This reduction reaches 65% when randomness is considered.
  • GAN Performance: SimSwap was found to be the most effective deepfake generator among those evaluated, producing high-quality results with minimal artifacts.
  • Metric Effectiveness: Face embedding distance was identified as the most effective metric for selecting dissimilar target faces for anonymization.

Limitations and Future Work

  • Dataset Dependency: The diversity and dissimilarity of generated fakes depend on the distribution of attributes within the synthetic dataset used for target queries.
  • Face Detector Limitations: Small or oblique faces might be missed by the face detector, potentially leaving them unanonymized.
  • Storage for Large Social Graphs: Users with thousands of friends might face significant client-side storage requirements for face embeddings.
  • Future Work: The authors suggest developing a secure client-server protocol for managing face embeddings for users with extensive social networks. They also note that further research into the diversity and dissimilarity of fakes based on dataset distribution and face resolution is warranted.

Summary updated

BibTeX
@inproceedings{ciftci2023my,
  author = {Umur A. Çiftçi and Gokturk Yuksek and İlke Demir},
  title = {{My Face My Choice: Privacy Enhancing Deepfakes for Social Media Anonymization}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2023},
}
Figure