Abstract
Generative Adversarial Networks (GANs) can generate hyperrealistic face images of synthetic identities based on a latent understanding of real images from a large training set. Despite their proficiency, the term "synthetic identity" remains ambiguous, and the uniqueness of the faces GANs produce is rarely assessed. Recent studies have found that identities from the training data can unintentionally appear in the faces generated by StyleGAN2, but the cause of this phenomenon is unclear. In this work, we propose a novel framework, SynthProv, that utilizes the improved interpolation ability of StyleGAN2 latent space and employs image composition to analyze leakage. This is the first method that goes beyond detection and traces the source or provenance of constituent identity signals in the generated image. Experiments show that SynthProv succeeds in both detection and provenance tasks using multiple matching strategies. We identify identities from FFHQ and CelebA-HQ training datasets with the highest leakage into the latent space as "leaking reals". Analyzing latent space behavior to evaluate generative model privacy via leakage is an important research direction, as undetected leaking reals pose a significant threat to training data privacy.
Introduction: The Problem of "Leaking Reals"
Generative Adversarial Networks (GANs), particularly architectures like StyleGAN2, produce incredibly realistic human faces. While these are often called "synthetic identities," they are learned from massive datasets of real people, such as FFHQ (70,000 images) or CelebA-HQ (30,000 images).
Recent research has highlighted a significant privacy concern: "leakage." Features of real individuals from the training set often manifest in synthetic outputs. While previous work focused on detecting this leakage, the authors of SynthProv argue that detection is insufficient. To truly understand and mitigate privacy risks, we need provenance—the ability to trace a synthetic face's features back to the specific real-world training samples that influenced them.
As shown in Figure 1, the framework identifies "leaking reals" (LRs)—training images whose identity information has been memorized or overly prioritized by the GAN, causing it to "bleed" into the synthetic manifold.
Methodology: The SynthProv Framework
The authors introduce SynthProv, an interpretable framework built on three main pillars:
1. Identity Composites
Instead of analyzing single images, the authors create "composites" by averaging latent vectors in the $w$ space. They found that these composites aggregate identity signals, making the "leakage" from training data more pronounced and easier to analyze than in individual samples.
2. Identity-Invariant Latent Directions
The team identifies a specific direction ($d^*$) in the StyleGAN2 latent space that corresponds to identity-salient features. By measuring distances perpendicular to this direction, they calculate a "latent identity distance" that is more robust than standard Euclidean metrics.
3. The Retrieval Pipeline
The core of SynthProv is its retrieval pipeline, detailed in Figure 3.
The process works as follows:
- Assistant Selection: For a given synthetic composite query ($Q$), the system selects "assistants" ($A$)—other synthetic images that share similar identity signals.
- Mixed Scoring: It then calculates a Mixed Score by combining dissimilarity scores from both a traditional face-matcher ($\mathcal{M}_e$, e.g., ArcFace) and the novel latent identity distance ($\mathcal{M}_l$).
- Provenance Identification: This score is used to rank images from the training set ($\mathcal{R}$), identifying the top "Leaking Reals" (LRs) that contributed to the synthetic face.
Key Insights and Results
One of the most significant insights is that "transitive aggregation" occurs in latent space. When creating a composite of several synthetic images, the shared leakage from real training samples becomes denser and more detectable than in a single generated image.
Quantitative results support this:
- Higher Embedding Density: Synthetic composites have a significantly higher "density" of real-image neighbors in embedding space compared to randomly sampled synthetic images. For FFHQ, the density score for composites was 5.20 vs 3.13 for standard samples.
- Empirical Proof of Leakage: Quantitative match scores showed that synthetic composites are often more similar to their "leaking real" sources than real composites are to their own parents. This suggests that GANs are sometimes "over-fitting" certain identities.
- Qualitative Success: The framework successfully retrieved real training images that share striking, coherent visual identity features with synthetic queries.
Limitations and Future Outlook
While SynthProv is a powerful tool for auditing generative model privacy, it has limitations:
- Architecture Specificity: The current framework is heavily optimized for StyleGAN2. When tested on older architectures like PGGAN, the results were less impressive because PGGAN’s latent space is less "semantically dense."
- Computational Overhead: Pre-computing latent identity distances for large datasets is time-intensive (taking ~14.5 hours in the authors' setup).
- Diffusion Models: While diffusion models also suffer from data replication, SynthProv was not tested on them. Extending this provenance logic to architectures like Stable Diffusion is a vital next step.
In conclusion, SynthProv moves the needle from simply knowing that GANs "cheat" by using training data, to knowing exactly who they are copying, providing a vital tool for auditing generative model privacy.
Code is available at https://github.com/jaisidhsingh/SynthProv.
Summary updated