Federated f-Differential Privacy.

Qinqing Zheng, Shuxiao Chen, Qi Long, Weijie SuPubMed 2021

TL;DR

This paper introduces "federated f-differential privacy," a new privacy definition for federated learning that protects individual data records, not just entire datasets. They propose a flexible framework, PriFedSync, that can implement various federated learning algorithms while provably achieving this new privacy guarantee, demonstrating a trade-off between privacy and model performance.

What Problem Does This Paper Address?

Federated learning (FL) allows multiple clients to collaboratively train machine learning models without sharing their raw data. While this offers a degree of privacy, a malicious client could potentially infer information about other clients' data by analyzing the shared model updates. Existing privacy solutions often focus on protecting an entire client's dataset, which might be too strong or not address the specific threat model where individual data records are the primary concern. This paper addresses the need for a more granular privacy guarantee at the record level in federated learning, considering scenarios where individual data points (like a patient's medical record or a user's search history) need protection, even if the client's participation in the federation is not sensitive.

Methods

The core of this work lies in introducing a novel privacy notion: federated f-differential privacy. This definition is built upon the framework of Gaussian differential privacy (GDP) and operates at the record level. This means it guarantees that an adversary cannot reliably determine the presence or absence of any single data record within a client's dataset. They define two variants: weak federated f-differential privacy (protecting against individual adversaries) and strong federated f-differential privacy (protecting against colluding adversaries).

To implement this, they propose a generic framework called PriFedSync. This framework is designed to be compatible with a wide range of existing federated learning algorithms. PriFedSync doesn't assume a trusted central aggregator and can handle both homogeneous (all clients have similar data distributions) and heterogeneous (clients have different data distributions) settings. The privacy analysis of PriFedSync leverages the composition theorems of GDP to provide provable privacy guarantees. For local training within PriFedSync, they suggest using noisy SGD, which, despite being computationally slower, often leads to better accuracy.

Datasets

The paper uses standard computer vision datasets for empirical evaluation:

  • MNIST: A dataset of handwritten digits.
  • CIFAR-10: A dataset of object images.

The authors simulate non-IID (non-independent and identically distributed) data distributions across clients for both datasets to better reflect real-world scenarios where data is often unevenly distributed.

Key Insights and Ideas

  • Record-level privacy is crucial for FL: Protecting individual data records is a more practical and relevant privacy concern in many federated learning applications than protecting entire datasets.
  • Federated f-differential privacy offers fine-grained protection: This new definition provides a mathematically rigorous way to quantify privacy at the individual data point level.
  • PriFedSync is a versatile framework: It can accommodate various FL algorithms and settings, making it a valuable tool for developing private FL systems.
  • Personalization can improve accuracy in heterogeneous settings: When client data is non-IID, personalized models trained within PriFedSync significantly outperform global models.
  • There's a clear privacy-utility trade-off: Increasing privacy (e.g., by adding more noise or using a smaller client sampling rate) generally leads to a decrease in model accuracy.

Key Results

  • Provable Privacy Guarantees: PriFedSync is shown to provably achieve federated f-differential privacy under various settings.
  • Accuracy Gains from Personalization: In non-IID MNIST experiments, personalized models achieved significantly better test accuracy compared to global models, especially under higher noise levels. For instance, Figure 1 shows personalized models outperforming global models for both $\sigma=0.75$ and $\sigma=1.0$.
  • Privacy-Computation Trade-off: Table 2 demonstrates that smaller batch sizes lead to better privacy (smaller $\mu_{max}$) but require more training iterations and total samples to reach a target accuracy. For example, reducing batch size from 16 to 8 roughly improved privacy by 0.83x but increased iterations by 2.78x.
  • Impact of Client Sampling: Figure 5 shows that a larger client sampling rate leads to faster convergence in MNIST experiments.

Limitations and Future Work

  • Computational Cost: The paper notes that using DP-SGD for local training, while beneficial for accuracy, is computationally slower due to per-sample gradient clipping.
  • Trade-offs: The empirical results clearly illustrate the inherent trade-offs between privacy, accuracy, and computation, which are central to designing practical FL systems.
  • Future Work: The paper suggests exploring more sophisticated constructions for personalized helper models within the PriFedSync framework and further investigating the privacy-computation trade-offs. The analysis of the privacy guarantee could potentially be tightened by incorporating prior knowledge of the helper model transformation function $F_i$.

Summary updated

Abstract

Federated learning (FL) is a training paradigm where the clients collaboratively learn models by repeatedly sharing information without compromising much on the privacy of their local sensitive data. In this paper, we introduce <i>federated f-differential privacy</i>, a new notion specifically tailored to the federated setting, based on the framework of Gaussian differential privacy. Federated <i>f</i>-differential privacy operates on <i>record level</i>: it provides the privacy guarantee on each individual record of one client's data against adversaries. We then propose a generic private federated learning framework PriFedSync that accommodates a large family of state-of-the-art FL algorithms, which provably achieves federated <i>f</i>-differential privacy. Finally, we empirically demonstrate the trade-off between privacy guarantee and prediction performance for models trained by PriFedSync in computer vision tasks.
BibTeX
@inproceedings{zheng2021federated,
  author = {Qinqing Zheng and Shuxiao Chen and Qi Long and Weijie Su},
  title = {{Federated <i>f</i>-Differential Privacy.}},
  booktitle = {PubMed},
  year = {2021},
}
Figure