Subgroup Membership Inference Audits of Differentially Private Synthetic Text

Yidan Sun, Viktor Schlegel, Srinivasan Nandakumar, Siew Kei Lam, Anil Anthony BharatharXiv 2026 · arXiv

TL;DR

Standard privacy audits for synthetic data focus on average-case risk, masking severe privacy threats to vulnerable subgroups. This paper introduces a subgroup-targeted membership inference framework, revealing that Differential Privacy (DP) offers uneven protection—leaving atypical records disproportionately exposed, with 40% of residual leakage concentrated in just 10% of the data.

What Problem Does This Paper Address?

Releasing synthetic text data instead of real datasets is a popular strategy to preserve privacy in sensitive domains like healthcare, finance, and law. To guarantee privacy, practitioners often apply Differential Privacy (DP).

However, standard methods for auditing these releases via Membership Inference Attacks (MIAs) only evaluate average-case risk across randomly selected records. This approach creates a dangerous blind spot: it can conceal high privacy risks for specific, vulnerable subgroups (such as geometric outliers or records with rare label combinations). This paper addresses this gap by formalizing a way to audit subgroup-specific privacy leakage in synthetic text generation.

Methods

The authors propose a Pool-Parameterized Membership Inference Game, where the target pool of records is an explicit parameter (e.g., outliers, rare labels, or random controls).

Overview of the pool-parameterized membership game, audit design, and empirical findings

The audit framework evaluates 32 distinct proxy metrics (spanning lexical features like n-gram overlap and embedding-space distances) across three attacker scenarios with increasing levels of background knowledge:

  1. Release-only: The attacker only sees the synthetic data release.
  2. Raw-reference: The attacker has access to auxiliary data from the same population.
  3. Generator-assisted: The attacker can additionally query the synthetic data generator.

The audit uses a two-stage protocol: first screening via within-pool games to identify vulnerable cells, and then adjudicating using a merged-pool audit that scores different record types against shared negatives. They test three distinct DP text generation mechanisms: DP-SGD fine-tuning (DP-Gen), API-based prompting (Aug-PE), and activation steering (EPSVec).

Datasets

The framework was evaluated across four datasets spanning healthcare, finance, and legal domains:

  • N2C2: Clinical discharge summaries annotated for obesity and co-morbidities.
  • PsyTAR: Social media posts labeled for adverse drug effects.
  • DMSAFN: Financial news text.
  • EurLEX: EU legislative documents annotated with EuroVoc concepts.
Corpus-level statistics for the evaluated datasets

Key Insights and Ideas

  • Uneven DP Protection: While DP successfully drops the average privacy leakage, it protects typical records much better than atypical ones. The noise introduced by DP strips away membership evidence from random records far more effectively than from outliers.
  • Leakage Concentration: Privacy risk under DP is not spread out evenly; it is highly concentrated. A small fraction of records carries the vast majority of the residual risk.
  • Mechanism-Bound Risk: Which records are vulnerable is a property of the specific release mechanism (the generator model and privacy budget), not an intrinsic property of the record itself. A record that leaks heavily under one generator might be perfectly safe under another, meaning data holders cannot simply "triage" data privacy risks beforehand based on text attributes alone.

Key Results

  • Without DP, fine-tuned generators act almost as membership oracles for high-risk subgroups, achieving high Area Under the ROC Curve (AUC) scores of 0.79 to 0.83.
  • Applying DP significantly reduces aggregate leakage, but subgroup membership remains inferable above chance across all tested budgets, with best-proxy AUCs remaining between 0.55 and 0.65.
  • Under DP, the top decile of records carries roughly 40% (and up to 72% in extreme cases) of the total positive vulnerability mass ($S_{10} \approx 40\%$).
Sorted per-record vulnerability showing risk concentration in the top decile
  • Traditional release-based attacks (like CANARY or DOMIAS) systematically underestimate this subgroup leakage because they rely on a single fixed signal rather than dynamically selecting the most informative proxy for a given release mechanism.

Limitations and Future Work

  • Scale: The evaluated corpora are relatively small (ranging from 604 to 5,102 records), and target pools are limited to 16–71 records.
  • Computational Cost: Running the full pipeline over multiple pool draws is extremely expensive; a single configuration of a pool, generator, and privacy budget takes 27–30 hours to run on an L40S GPU, limiting the ability to capture full pool-drawing variance.
  • Threat Model: The adversary model is strictly release-based. Future work could explore how these subgroup risks evolve if the attacker gains direct model access or if larger datasets are used.

Summary updated

Abstract

Synthetic data releases are increasingly proposed in the literature as a means of sharing realistic data replicas in lieu of sensitive private datasets. Even when the worst-case privacy leakage of such releases is bounded by means of differential privacy (DP), in practice a residual risk remains. Membership inference attack (MIA) audits are conducted to empirically quantify this risk. However, existing methods only measure average-case risk for randomly drawn records, which might conceal the risk to vulnerable subgroups. To highlight this issue, we define a subgroup-targeted membership inference game in which the target pool is an explicit parameter, and instantiate it with an audit of 32 proxies under three scenarios with different levels of attacker knowledge, across four datasets, three generators (DP-SGD fine-tuning, API-based prompting, and activation steering), and five privacy budgets. The audit shows that synthetic releases leak subgroup membership and that prior attacks systematically underestimate this leakage. DP is effective at the aggregate level: it substantially reduces average leakage at every budget we test. Three observations temper this picture. First, the remaining leakage is concentrated rather than spread out: under DP, a tenth of the records carries roughly 40% of it. Second, the protection DP delivers in practice is uneven: within its worst-case guarantee, the noise removes more of the measured leakage from random records than from high-risk ones---and a merged-pool audit that scores both record types against shared negatives confirms this at the record level. Third, \emph{which} records leak proves to be a property of the release mechanism rather than of the record alone, so record-level risk cannot be assessed independently of the release.
BibTeX
@misc{sun2026subgroup,
  author = {Yidan Sun and Viktor Schlegel and Srinivasan Nandakumar and Siew Kei Lam and Anil Anthony Bharath},
  title = {{Subgroup Membership Inference Audits of Differentially Private Synthetic Text}},
  year = {2026},
  eprint = {2609.09848},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.09848},
}
Figure