Introduction: The Free-Rider Problem in Distributed GANs
Generative Adversarial Networks (GANs) have become a cornerstone of synthetic data generation. However, privacy regulations often necessitate distributed training, where data remains siloed. Multi-Discriminator GANs (MD-GANs) address this by employing a central generator that interacts with multiple distributed discriminators, each holding private datasets.
A significant vulnerability in this architecture is the presence of "free-riders." These participants aim to benefit from the final generator model without contributing real data or computational effort. In an MD-GAN, a free-rider typically uses a randomly initialized discriminator. Unlike passive leeches in other systems, these actors provide noisy, random feedback that actively degrades the generator's performance, leading to poor image quality and higher Fréchet Inception Distance (FID) scores.
Proposed Defenses: DFG and DFG+
To mitigate these attacks, the authors propose two defense mechanisms: Discriminator Filtering for Generator (DFG) and its extension, DFG+.
DFG: Discriminator Filtering for Generator
The DFG mechanism relies on periodic probing and clustering:
- Probing: The generator periodically sends a set of synthetic images to all participating discriminators.
- Local Detector: The generator maintains a local "detector"—a discriminator intentionally initialized with random weights to mimic free-rider behavior.
- Clustering: The generator collects classification responses from all participants and applies 2-means clustering. Benign discriminators, having been trained on real data, produce similar responses that differ significantly from the random responses of the detector and free-riders. The generator then identifies and ignores the cluster containing its own detector.
DFG+: Securing Peer Swapping
DFG+ extends this defense to MD-GAN variants where discriminators swap model weights to prevent overfitting. It enables benign discriminators to identify and reject swaps with suspected free-riders, preventing "model leakage" where attackers might steal well-trained models from peers.
Experimental Evaluation
The defenses were evaluated on MNIST, CIFAR-10, and CIFAR-100 datasets with 5 benign clients and varying numbers of free-riders (0 to 5).
Key Results
- Detection Accuracy: DFG and DFG+ achieved 100% precision and recall across all CIFAR-10 and CIFAR-100 scenarios, perfectly identifying free-riders without penalizing benign participants.
- Performance Restoration: On CIFAR-10, the defense improved FID scores by 5.22% to 11.53%. On the more complex CIFAR-100, improvements ranged from 5.79% to 13.22%.
- Computational Efficiency: The 2-means clustering on a small probing set introduces negligible overhead compared to the primary training tasks.
Key Insights
The research highlights that free-riding is not merely a resource issue but a destructive force. In systems utilizing peer-swapping, free-riders can effectively "poison" the training pool, causing the generator to produce images that are heavily degraded by noise.
The use of a local "detector" as an anchor is a particularly effective strategy. By creating a known "bad" reference point, the system avoids the need to define "good" behavior explicitly, instead identifying outliers relative to a known malicious baseline.
Conclusion and Future Directions
DFG and DFG+ provide a robust, efficient defense against free-riders in distributed GAN training. While highly effective against random-initialization attacks, future work should address adaptive adversaries who might attempt to mimic benign responses, as well as the potential for collusion among multiple free-riders. Testing on higher-resolution datasets like ImageNet will also be crucial for validating scalability.
Summary updated