Federated Generative Learning with Foundation Models
Lists
Only you see this
Abstract
Existing federated learning solutions focus on transmitting features, parameters, or gradients between clients and server, which suffer from serious low-efficiency and privacy-leakage problems. Thanks to the emerging foundation generative models, we propose a novel federated learning framework, namely Federated Generative Learning. In this framework, each client can create text prompts that are tailored to their local data, based on its features, and then send them to the server. Given the received prompts, the informative training data can be remotely synthesized on the server using foundation generative models. This new framework offers several advantages, including enhanced communication efficiency, improved resilience to distribution shift, significant performance gains, and enhanced privacy protection. We validate these benefits through extensive experiments conducted on ImageNet and DomainNet datasets, e.g., on ImageNet100 dataset, with a highly skewed data distribution, our method outperforms FedAvg by 12% in a single communication round. Moreover, our approach only requires 229 Bytes prompts for communication, while FedAvg necessitates the transmission of 42.7 MB parameters.
BibTeX
@misc{zhang2023federated,
author = {Jie Zhang and Xiao hua Qi and Shengyuan Pang and Siyuan Pan and Xiaobing Tu and Pengfei Wan and Bo Zhao},
title = {{Federated Generative Learning with Foundation Models}},
year = {2023},
eprint = {2306.16064},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2306.16064},
}