Privacy Amplification by Subsampling in Differential Privacy

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2212 papers331 conceptsfrozen 2026-10-04

In the field of privacy-preserving machine learning, there is a rare "free lunch" known as privacy amplification by subsampling. The core intuition is simple: if an algorithm only examines a small, random fraction of a dataset, the privacy risk to any single individual in that dataset is naturally diluted.

This mechanism has become the cornerstone of modern Differentially Private Stochastic Gradient Descent (DP-SGD), allowing researchers to train high-utility models under strict privacy budgets by leveraging the inherent randomness of mini-batch selection.

The Intuition: Hiding in the Unselected

Differential Privacy (DP) typically requires adding noise to a computation to mask the influence of any single record. However, as Bing-Rong Lin and Shantanu Rane demonstrate in On the Benefits of Sampling in Privacy Preserving Statistical Analysis on Distributed Databases, "less is more." If an individual's data is not even sampled for a specific computation, their privacy loss for that step is zero.

When we sample a subset of data with probability $q$, an adversary faces a fundamental uncertainty: they do not know if a specific individual was even included in the batch. Dong Su showed in Provably Private Data Anonymization that this uncertainty mathematically scales the privacy loss parameter $\epsilon$. Effectively, if an algorithm is $\epsilon$-private on a subset, it becomes roughly $q\epsilon$-private over the whole population. This allows practitioners to use much less additive noise than would be required if the entire dataset were processed at once.

Sampling as a Primary Privacy Lever

While traditional DP research focuses on the magnitude of noise, recent evidence suggests that the sampling rate might be a more potent lever for protection. Franziska Boenisch and Nicolas Papernot argue in From Differential Privacy to Bounds on Membership Inference: Less can be More that training on a small subset with minimal noise often yields a superior accuracy-privacy trade-off compared to training on a full dataset with high noise.

This shift in perspective treats the sampling rate $q$ as a primary control variable. By reducing $q$, researchers can mitigate Membership Inference Attacks more effectively than by simply increasing the noise parameter in DP-SGD.

The Implementation Gap: Poisson vs. Fixed-Size

The theoretical "gold standard" for amplification is Poisson subsampling, where each record is included in a batch independently with probability $q$. However, Poisson sampling results in variable batch sizes, which wreaks havoc on modern hardware like GPUs that demand static tensor shapes.

To bridge this gap, researchers have explored several alternatives:

Advanced Accounting and Group Privacy

The mathematical complexity of tracking these gains has led to the development of sophisticated "privacy accountants." Yu-Xiang Wang and Borja Balle provided the foundational Analytical Moments Accountant for subsampled Rényi Differential Privacy, which allows for tight tracking of privacy loss over thousands of training steps.

Furthermore, subsampling provides a unique benefit for group privacy. Standard DP bounds for groups (e.g., all data from one family) usually degrade linearly with group size. However, Jan Schuchardt shows in Group Privacy Amplification and Unified Amplification by Subsampling that a joint analysis of sampling and group structure captures the statistical improbability of an entire group being sampled in a single small batch, leading to much tighter and more practical privacy guarantees.

Applications Beyond SGD

The principle of amplification is being extended into diverse domains:

Papers in this version

Showing 60 of 212 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 168 papers
  2. Version 1 · 2026-10-04 · 212 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_privacy_amplification_by_subsampling_a64cd070,
  title = {privacy amplification by subsampling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/privacy-amplification-by-subsampling/rdr_a64cd07035b947729ba7b3087601ca28},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_privacy_amplification_by_subsampling_in__a64cd070_v1,
  title = {Privacy Amplification by Subsampling in Differential Privacy},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/privacy-amplification-by-subsampling-in-differential-privacy/rdr_a64cd07035b947729ba7b3087601ca28/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}