Membership Inference Attack on Private Evolution

Daniil Filienko, Sikha Pentyala, Martine De CockarXiv 2026 · arXiv

TL;DR

This paper introduces EVO-MIA, the first white-box Membership Inference Attack (MIA) specifically designed for the Private Evolution (PE) framework. By exploiting the "evolutionary traces" left by private data during iterative voting processes, the authors demonstrate that even synthetic data generated with formal Differential Privacy (DP) guarantees can leak information about whether a specific individual's record was used in the generation process.

What Problem Does This Paper Address?

Synthetic Data Generation (SDG) is a popular way to share data insights without sharing the raw, sensitive data itself. A recent and efficient approach is Private Evolution (PE). Unlike traditional methods that require expensive fine-tuning of large models on private data, PE uses "inference-only" access to pre-trained foundation models (like LLMs).

In PE, the data holder prompts a model to generate synthetic candidates. Private data points then "vote" for the candidates most similar to them. Over multiple rounds, the synthetic distribution evolves to match the real data. While PE includes Differential Privacy (DP) mechanisms to protect these votes, the authors argue that formal guarantees aren't enough—we need empirical "stress tests" to see if an adversary can still figure out if a specific person was in the dataset.

Methods

The authors propose EVO-MIA (Membership Inference via Evolutionary Traces). The core idea is that every private instance leaves a signal by "nudging" up the vote count of its closest synthetic neighbor.

Figure 1: Overview of the voting mechanism and the proposed MIA. Private instances cast votes for the synthetic candidates. A noisy vote histogram over these candidates is released. The adversary tests if the target instance x⋆nudges the histogram, ultimately classifying it as a member or non-member.

As shown in Figure 1, the attack works as follows:

  1. Target Identification: For a target instance $x^*$, the adversary finds the closest synthetic candidate in each round.
  2. Likelihood Estimation: Using an auxiliary "reference" dataset (publicly available data from a similar distribution), the adversary estimates how many votes a candidate should get if the target $x^*$ were not a member.
  3. Log-Likelihood Ratio: The attack calculates the probability of the observed (noisy) vote count under two hypotheses: $H_1$ (the target is a member) and $H_0$ (the target is not).
  4. Accumulation: This score is summed across all evolution rounds. A high cumulative score suggests the target was indeed a member.

The authors also introduce EVO-MIA-BB, a "black-box" variant that works even if the adversary only sees which candidates survived to the next round, rather than the raw vote counts.

Datasets

The attack was evaluated across three modalities using a variety of standard and specialized datasets:

  • Text: OpenReview (peer reviews) and MIMIC-III (hospital discharge notes).
  • Images: CIFAR-10, Camelyon17 (histopathology), and AFHQ Cat faces.
  • Tabular: Adult (income), Artificial-characters, Breast-cancer, and Person-activity.
Table 4: Datasets used in our evaluation, grouped by modality, with train subset from which members are samples, test set which constitutes non-members, and reference set sizes

Key Insights and Ideas

  • Iterative Filtering is a Double-Edged Sword: While the iterative nature of PE helps synthetic data converge to high quality, it also provides the adversary with multiple "looks" at how the private data influences the model, effectively amplifying the leakage signal.
  • High Dimensionality Increases Risk: The attack performed significantly better on text and image data than on tabular data. In high-dimensional spaces, synthetic candidates are more likely to be "close" to only a few specific private points, making the "nudge" in vote counts much more distinct.
  • Utility $\neq$ Privacy: A surprising finding was that low utility (poor quality synthetic data) does not guarantee low leakage. For the OpenReview text dataset at $\epsilon=10$, the synthetic data had very low utility but still retained a high membership leakage (AUC of 0.685).
  • Dataset Size Matters: On very large datasets (like 'adult' or 'person-activity'), the attack was less effective. This is because a single member's vote is a "drop in the bucket" compared to the total expected votes for a candidate cell.

Key Results

EVO-MIA consistently outperformed existing state-of-the-art black-box attacks (like LOGAN for images or n-gram MIAs for text).

  • Text: On OpenReview, EVO-MIA achieved an AUC of 0.873 (where 0.5 is random guessing) in the non-private setting ($\epsilon=\infty$) and remained potent at 0.685 even with a privacy budget of $\epsilon=10$.
  • Images: For the Cat dataset, the attack reached an AUC of 0.846 at $\epsilon=\infty$.
  • Tabular: While less effective on large tables, it still slightly surpassed state-of-the-art baselines on the breast-cancer dataset.
Table 3: MIA results for PE-text

Limitations and Future Work

The primary limitation is the threat model: the main version of the attack assumes the adversary can see the noisy vote histograms released in each round. While the authors' black-box variant (EVO-MIA-BB) mitigates this by only requiring survival data, keeping intermediate rounds completely private would significantly reduce the attack's effectiveness.

For future work, the authors suggest exploring even tighter privacy auditing and using LLM-based agents to discover even stronger membership signals in the evolutionary process.

Summary updated

Abstract

Synthetic Data Generation (SDG) is an actively studied way to enable safe sharing of private data more broadly. Inference-only SDG methods, such as Private Evolution (PE), are particularly appealing, as they can leverage black-box access to pre-trained foundation models to generate high quality data without the need for model fine-tuning on the private data. PE is explicitly designed to produce synthetic data under Differential Privacy (DP). Yet formal DP guarantees do not eliminate the need for empirical privacy auditing of the practical susceptibility of released synthetic data to adversarial attacks. In this work, we introduce the first white-box Membership Inference Attack (MIA) tailored to the voting-based algorithm underlying the PE framework. By exploiting how the iterative filtering process biases the survival and proximity of synthetic candidates toward private instances, our attack achieves empirical performance indicating private data leakage in the PE applications across image, text, and tabular modalities.
BibTeX
@misc{filienko2026membership,
  author = {Daniil Filienko and Sikha Pentyala and Martine De Cock},
  title = {{Membership Inference Attack on Private Evolution}},
  year = {2026},
  eprint = {2610.30514},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.30514},
}
Figure