Learning to Generate Image Embeddings with User-Level Differential Privacy

Zheng Xu, Maxwell D. Collins, Yuxiao Wang, Liviu Panait, Sewoong Oh, Sean Augenstein, Ting Liu, Florian Schroff, H. Brendan McMahan

Introduction

Training machine learning models on sensitive visual data, such as facial images or personal landmarks, presents significant privacy risks. Models are prone to "memorizing" specific training examples, which can lead to data leakage. Differential Privacy (DP) offers a rigorous mathematical framework to mitigate these risks. However, applying DP to large-scale image embedding models—where the goal is to map images into a feature space for tasks like recognition—has historically been challenging due to the massive scale of the classification space.

The Challenge of DP in Large-Scale Embeddings

In tasks like face recognition, the number of unique identities (classes) can reach into the millions. Standard DP methods, such as DP-SGD, add noise to the entire model's gradients. In these architectures, the final classification layer (the "head") grows linearly with the number of classes. With millions of users, this layer becomes extremely large, leading to sparse updates. When DP noise is added to such a massive, sparse layer, the signal-to-noise ratio drops precipitously, often causing the model's accuracy to collapse.

The DP-FedEmb Framework

To address the "exploding head" problem, researchers at Google developed DP-FedEmb. This framework adapts Federated Learning (FL) principles for centralized data to protect user privacy while maintaining model utility.

The DP-FedEmb training pipeline: backbone-head partitioning and private aggregation.

The core innovation of DP-FedEmb lies in its architectural partitioning and aggregation strategy:

  • Backbone-Head Partitioning: The model is split into a backbone (feature extractor) and a head (classification layer).
  • Partial Aggregation: Only the updates to the backbone are aggregated, clipped, and noised for the DP release. The classification heads are treated as local, temporary entities that are discarded after each round, preventing the identity-specific head from inflating noise requirements.
  • Virtual Clients: Users are grouped into "virtual clients" to increase data diversity per update and stabilize the training process, especially when individual users have few images.
  • Public Pretraining: The backbone is initialized using non-sensitive public data, providing a strong starting point for private fine-tuning.
  • Local Fine-tuning: Virtual clients perform multiple steps of local SGD on both the backbone and their local heads before sending backbone updates back for private aggregation.

Experimental Setup

The researchers evaluated DP-FedEmb using standard architectures like ResNet-50 and MobileNetV2 across several diverse benchmarks including DigiFace (synthetic faces), EMNIST (character recognition), Google Landmarks (GLD), and iNaturalist (iNat).

Key Results and Insights

The study yielded several critical insights into the relationship between scale and privacy:

  • Superior Utility: DP-FedEmb consistently outperformed the standard DP-FedAvg baseline. On the DigiFace dataset, it achieved significantly higher recall within the same privacy budget.
  • The 5% Threshold: For a simulated population of 10 million users, the framework achieved a strong privacy guarantee ($\epsilon < 4$) with less than a 5% drop in accuracy compared to a non-private model.
  • Scale as a Privacy Enabler: A key finding is that DP-FedEmb becomes more effective as the number of users increases. In large-scale settings, the noise added for privacy becomes a smaller fraction of the total aggregated signal.
  • Feasibility of Large Models: This work represents the first successful training of a ResNet-50 (~23 million parameters) with meaningful user-level DP in an embedding context.

Conclusion

DP-FedEmb demonstrates that high-utility image embedding models can be trained with strong user-level differential privacy by focusing protection on the model backbone and treating classification heads as transient. While the method requires careful hyperparameter tuning and benefits significantly from large datasets, it provides a scalable path forward for privacy-preserving computer vision.

Summary updated

Abstract

Small on-device models have been successfully trained with user-level differential privacy (DP) for next word prediction and image classification tasks in the past. However, existing methods can fail when directly applied to learn embedding models using supervised training data with a large class space. To achieve user-level DP for large image-to-embedding feature extractors, we propose DP-FedEmb, a variant of federated learning algorithms with per-user sensitivity control and noise addition, to train from user-partitioned data centralized in the datacenter. DP-FedEmb combines virtual clients, partial aggregation, private local fine-tuning, and public pretraining to achieve strong privacy utility trade-offs. We apply DP-FedEmb to train image embedding models for faces, landmarks and natural species, and demonstrate its superior utility under same privacy budget on benchmark datasets DigiFace, EMNIST, GLD and iNaturalist. We further illustrate it is possible to achieve strong user-level DP guarantees of $ε<4$ while controlling the utility drop within 5%, when millions of users can participate in training.
BibTeX
@inproceedings{xu2022learning,
  author = {Zheng Xu and Maxwell D. Collins and Yuxiao Wang and Liviu Panait and Sewoong Oh and Sean Augenstein and Ting Liu and Florian Schroff and H. Brendan McMahan},
  title = {{Learning to Generate Image Embeddings with User-Level Differential Privacy}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2022},
}
Figure