Real-World Benchmarks Make Membership Inference Attacks Fail on Diffusion Models

Chumeng Liang, Jiaxuan YouSubmission to ICLR 2025 · Rejected

TL;DR

Existing Membership Inference Attacks (MIAs) against diffusion models appear remarkably successful only because their benchmarks rely on heavily over-trained models and distribution-shifted non-members. When evaluated on CopyMark—a new, realistic benchmark with properly matched distributions and foundation models trained for standard single epochs—state-of-the-art MIAs collapse toward random guessing, proving they are currently unfit to serve as reliable evidence in AI copyright disputes.

What Problem Does This Paper Address?

As artists and copyright holders initiate high-profile lawsuits against generative AI companies, Membership Inference Attacks (MIAs) have been touted as a promising auditing tool to prove whether a copyrighted image was used during pre-training.

However, the authors uncover two fatal flaws in how diffusion MIAs are currently evaluated:

  1. Severe Over-training: Prior benchmarks often evaluate models trained for hundreds or thousands of epochs over small datasets (e.g., CIFAR-10 or CelebA for 500+ epochs), causing memorization that does not reflect real foundation models (which typically train for just 1–2 epochs over billions of web images).
  2. Dataset Distribution Shift: Existing evaluations frequently pair members and non-members from fundamentally different sources (e.g., LAION-trained models tested against MS-COCO non-members). In these setups, MIAs inadvertently act as generic dataset classifiers distinguishing image domains rather than detecting true model membership.

Methods

To resolve these issues, the authors introduce CopyMark, the first unified, realistic MIA benchmark built directly on top of the Hugging Face diffusers library:

  • Zero/Minor Distribution Shifts: Non-members are drawn from identical or closely matched distributions (e.g., holdouts from the same multi-million image scrape).
  • Realistic Pre-trained Models: Evaluates real-world, large-scale models trained for only a single epoch.
  • Rigorous Two-Stage Evaluation: Unlike prior work that evaluates thresholds on the exact same dataset used to calibrate them, CopyMark splits data into a validation set (to tune detection thresholds or train classifiers) and a separate holdout test set for blind evaluation.
  • Shift Quantification: Quantifies dataset shift between members and non-members explicitly using Fréchet Distance (FD), Mahalanobis Distance, and Normalized Wasserstein Distance on CLIP embeddings, as well as an implicit linear probing classifier.
  • Blind Baseline: Evaluates a model-agnostic ConvNeXt image classifier trained directly on members vs. non-members without inspecting the diffusion model at all, isolating how much performance stems purely from dataset bias.

Evaluated attack families include loss-based MIAs (SecMI, PIA, PFAMI) and classifier-based MIAs (GSA₁, GSA₂).

Datasets

CopyMark evaluates five experimental setups:

  • Defective Reference Setups:
  • Setup (a): LDM on CelebA-HQ (members) vs. FFHQ (non-members) [overtrained for 500 epochs, severe shift].
  • Setup (b): Stable Diffusion v1.5 on LAION Aesthetic v2 5+ (members) vs. MS-COCO 2017 (non-members) [severe shift].
  • Novel Realistic Setups:
  • Setup (c): Stable Diffusion v1.5 on LAION members vs. LAION Multi Translated non-members (unshifted LAION-2B distribution).
  • Setup (d): CommonCanvas-XL-C on CommonCatalog (CC-BY) vs. MS-COCO 2017 (minor shift).
  • Setup (e): Kohaku-XL-Epsilon on HakuBooru anime dataset members vs. an unshifted HakuBooru holdout set.

Key Insights and Ideas

  • MIA "Success" is Mostly a Hallucination: On the flawed setups (a and b), attacks show high apparent performance. But the "Blind" baseline—which never touches the diffusion model—matches or outperforms them (achieving AUCs > 0.99). The attacks were merely picking up image style and domain shifts.
  • Direct Correlation Between Shift and Success: When interpolating non-member sets from 0% to 100% distribution-shifted images (COCO vs. LAION), attack metrics rise in lockstep with the Fréchet Distance.
  • Single-Epoch Training Leaves Faint Traces: Foundation models trained for a single epoch have nearly identical loss landscapes for member and non-member images, removing the training loss gap that standard MIAs rely on.
  • Generalization Gaps in Classifier-Based MIAs: While classifier-based methods (GSA) achieve near-perfect training accuracy on validation splits, their false positive rates explode on unseen test splits (up to 40%+), rendering them unusable for high-confidence forensics.

Key Results

  • Loss-Based MIAs Fail Completely: Under unshifted setups (c and e), loss-based methods (SecMI, PIA, PFAMI) drop to near-chance performance:
  • On Stable Diffusion v1.5 (setup c), AUCs plunge to 0.510 – 0.535 (down from ~0.70–0.76 in shifted setups). True Positive Rates at 1% False Positive Rate (TPR@1%FPR) collapse to just ~1.0%–1.5%.
  • On Kohaku-XL (setup e), PFAMI achieves 0% TPR at 1% FPR on the blind test set (AUC = 0.498).
  • Classifier-Based Attacks Overfit: On setup (c), GSA achieves 100% TPR on validation, but on the blind test set, the false positive rate spikes to 27.0%–41.8%.
  • Quantifying the "Golden" Threshold: Fréchet Distance (FD) on CLIP embeddings strongly reflects distribution shift. Real-world benchmarks require an FD around 0.036–0.059; setups with FD > 0.20 are heavily confounded by dataset bias.

Limitations and Future Work

  • Root Cause of Attack Failure: Diffusion losses depend heavily on specific timesteps and injected noise. Because pre-trained models see an image only once during training, randomly sampling noise and timesteps during an attack fails to pinpoint the actual optimized loss step. Future MIAs will need explicit modeling of timestep and noise trajectory dependencies.
  • Scope: The benchmark focuses primarily on Latent Diffusion Models (LDMs/SDXL); evaluating other architectures like Rectified Flow Transformers (e.g., SD3, Flux) remains future work.
  • Legal Ramifications: The authors emphasize that US and EU copyright frameworks require demonstrating "substantial similarity" between plaintiff works and model outputs. A binary membership indicator—even if perfected—does not fulfill legal requirements for copyright infringement, cautioning practitioners against over-relying on MIAs in court.

Summary updated

Abstract

Membership inference attacks (MIAs) on diffusion models have emerged as potential evidence of unauthorized data usage in training pre-trained diffusion models. These attacks aim to detect the presence of specific images in training datasets of diffusion models. Our study delves into the evaluation of state-of-the-art MIAs on diffusion models and reveals critical flaws and overly optimistic performance estimates in existing MIA evaluation. We introduce CopyMark, a more realistic MIA benchmark that distinguishes itself through the support for pre-trained diffusion models, unbiased datasets, and fair evaluation pipelines. Through extensive experiments, we demonstrate that the effectiveness of current MIA methods significantly degrades under these more practical conditions. Based on our results, we alert that MIA, in its current state, is not a reliable approach for identifying unauthorized data usage in pre-trained diffusion models. To the best of our knowledge, we are the first to discover the performance overestimation of MIAs on diffusion models and present a unified benchmark for more realistic evaluation.
BibTeX
@misc{liang2024real,
  author = {Chumeng Liang and Jiaxuan You},
  title = {{Real-World Benchmarks Make Membership Inference Attacks Fail on Diffusion Models}},
  year = {2024},
  eprint = {2410.03640},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.03640},
}

Loading related content...

Figure