Abstract
The Partial Disentanglement with Partially-Federated GANs (PaDPaF) framework addresses the challenge of training generative models across decentralized clients with heterogeneous data. By partitioning the model into shared global "content" modules and private local "style" modules, PaDPaF enables high-quality personalized data generation and robust feature extraction. The approach utilizes latent-contrastive regularization to ensure effective disentanglement, allowing for applications such as style transfer and data anonymization even in highly non-IID settings.
Introduction
Federated Learning (FL) typically focuses on classification tasks, but extending it to generative models like GANs is complicated by data heterogeneity. Variations in lighting, rotation, or sensor noise—collectively referred to as "style"—can cause local data distributions to diverge significantly. Standard FL methods often struggle with this non-IID data, leading to poor generalization or the loss of client-specific nuances.
PaDPaF tackles this by aiming for disentanglement: separating the core semantic content (e.g., the identity of a digit) from its specific stylistic manifestation. The goal is to learn a shared content representation across the federation while respecting the unique stylistic characteristics of individual clients without centralizing their private data.
Methods
The PaDPaF architecture splits the GAN components into Federated Modules, which are averaged globally (e.g., via FedAvg), and Private Modules, which remain local to each client.
Architecture and Federation
The "content" portions of the generator and discriminator are shared, while "style" components—such as style vectorizers and conditional batch normalization layers—are kept private. This allows the global model to learn universal features while local models specialize in client-specific rendering.
Latent-Contrastive Regularization
To enforce the separation of content and style, the authors employ a regularization term inspired by Barlow Twins. This loss encourages the discriminator to produce consistent representations for images sharing the same content latent, regardless of the style latent used during generation.
Causal Foundation
The framework is grounded in a causal model where client identity influences both content and style. Through federated averaging, the client-specific influence on the content mechanism is effectively marginalized, leaving a robust, global content representation.
Experimental Evaluation
The model was tested across several datasets designed to simulate real-world heterogeneity, including MNIST, CIFAR-10, and CelebA. In these setups, clients were assigned unique data augmentations or specific attributes (like "Mustaches" or "Eyeglasses") to represent distinct local styles.
Key Results
- Feature Quality: A linear classifier trained on unsupervised content features from PaDPaF achieved 98.1% accuracy on MNIST, indicating high-quality disentanglement.
- Generative Performance: PaDPaF achieved an average Frechet Inception Distance (FID) of 8.28, significantly outperforming baselines like Ditto (60.46).
- Disentanglement Verification: Style features were highly predictive of client identity (93.3% accuracy), whereas content features were not, confirming successful separation.
Discussion and Future Work
PaDPaF demonstrates that partial federation is a powerful tool for handling non-IID data in generative modeling. It enables "style transfer" and data anonymization by re-rendering content in a different client's style.
However, limitations remain. The current architecture sometimes struggles with geometric transformations like rotations, occasionally conflating them with content. Future research may explore more flexible generator architectures or server-side deployment strategies to further enhance privacy and robustness.
Summary updated