GeFL: Model-Agnostic Federated Learning with Generative Models

Honggu Kang, Seohyeon Cha, Joonhyuk Kang

Abstract

Federated learning (FL) is a promising paradigm in distributed learning while preserving the privacy of users. However, the increasing size of recent models makes it unaffordable for a few users to encompass the model. It leads the users to adopt heterogeneous models based on their diverse computing capabilities and network bandwidth. Correspondingly, FL with heterogeneous models should be addressed, given that FL typically involves training a single global model. In this paper, we propose Generative Model-Aided Federated Learning (GeFL), incorporating a generative model that aggregates global knowledge across users of heterogeneous models. Our experiments on various classification tasks demonstrate notable performance improvements of GeFL compared to baselines, as well as limitations in terms of privacy and scalability. To tackle these concerns, we introduce a novel framework, GeFL-F. It trains target networks aided by feature-generative models. We empirically demonstrate the consistent performance gains of GeFL-F, while demonstrating better privacy preservation and robustness to a large number of clients. Codes are available at [1].

1. Introduction

Standard Federated Learning (FL) typically relies on a "one-size-fits-all" approach where every client must train the exact same global model architecture. This requirement presents a significant bottleneck in real-world edge computing environments, where devices range from high-performance workstations to low-power IoT sensors. Forcing a uniform architecture either excludes resource-constrained devices or underutilizes powerful ones. Furthermore, clients may wish to use proprietary, pre-trained models that cannot be modified or disclosed.

Existing model-heterogeneous FL methods often require curated public datasets to align knowledge between different architectures, which is frequently unavailable in privacy-sensitive domains. GeFL addresses this by enabling clients with diverse model architectures to learn collaboratively without sharing raw data or relying on external datasets.

2. Methodology

The researchers propose two primary frameworks to enable model-agnostic learning:

2.1 The GeFL Framework

Instead of aggregating model weights, GeFL utilizes a Federated Generative Model (FedGen) to bridge knowledge. The process involves three stages: local training of generators on private data, server-side aggregation into a global generator, and knowledge transfer where the global generator synthesizes training samples for heterogeneous target models.

The GeFL framework: Generative knowledge aggregation at the server and model-heterogeneous training at the clients.

2.2 GeFL-F: Feature-Level Knowledge Aggregation

To enhance privacy and scalability, GeFL-F introduces a shared, lightweight feature extractor. Rather than generating full-resolution images, the generator produces lower-dimensional feature representations. This abstraction reduces the risk of sensitive data reconstruction and lowers communication overhead, making it more robust for large-scale client deployments.

3. Experimental Results and Discussion

The framework was evaluated on standard benchmarks including MNIST, CIFAR-10, SVHN, and CelebA.

3.1 Accuracy and Scalability

GeFL consistently outperformed baselines like FedAvg and FedProx in heterogeneous settings. On CIFAR-10, GeFL achieved approximately 58.45% accuracy, surpassing FedAvg's 55.65%. While the base GeFL faced challenges as client numbers increased, GeFL-F demonstrated superior stability. On the CelebA dataset with 100 clients, GeFL-F maintained high performance where other methods degraded.

3.2 Privacy Preservation

Privacy was measured using the Mean Nearest Neighbor Distance (MND) ratio. GeFL-F proved significantly more private, with an MND ratio of roughly 0.1 compared to 1.0 for the pixel-based GeFL. This indicates that the feature-level generator avoids "copy-pasting" training images, providing a natural privacy filter.

Synthetic face images generated by the model, illustrating the diversity of samples used for training.

3.3 The Role of Generative Diversity

A key finding is that for FL, sample diversity often outweighs visual quality. In experiments with Diffusion models, lower guidance scores—which produce noisier but more diverse images—led to better downstream accuracy. Additionally, GeFL is particularly effective in data-scarce scenarios, where synthetic data acts as a global booster to prevent local overfitting.

4. Conclusion

GeFL and its feature-level variant, GeFL-F, provide a robust solution for federated learning across heterogeneous devices. By using generative models as "knowledge translators," the framework removes the architectural constraints of traditional FL. Future work will focus on optimizing the local computational cost of training generative models and exploring decentralized aggregation methods to further enhance security and auditability.

[1] Code repository link (placeholder as per abstract).

Summary updated

Abstract

Federated learning (FL) is a promising paradigm in distributed learning while preserving the privacy of users. However, the increasing size of recent models makes it unaffordable for a few users to encompass the model. It leads the users to adopt heterogeneous models based on their diverse computing capabilities and network bandwidth. Correspondingly, FL with heterogeneous models should be addressed, given that FL typically involves training a single global model. In this paper, we propose Generative Model-Aided Federated Learning (GeFL), incorporating a generative model that aggregates global knowledge across users of heterogeneous models. Our experiments on various classification tasks demonstrate notable performance improvements of GeFL compared to baselines, as well as limitations in terms of privacy and scalability. To tackle these concerns, we introduce a novel framework, GeFL-F. It trains target networks aided by feature-generative models. We empirically demonstrate the consistent performance gains of GeFL-F, while demonstrating better privacy preservation and robustness to a large number of clients. Codes are available at [1].
BibTeX
@misc{kang2024gefl,
  author = {Honggu Kang and Seohyeon Cha and Joonhyuk Kang},
  title = {{GeFL: Model-Agnostic Federated Learning with Generative Models}},
  year = {2024},
  eprint = {2412.18460},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2412.18460},
}
Figure