FedDrop: Trajectory-weighted Dropout for Efficient Federated Learning
Lists
Only you see this
Abstract
Federated learning (FL) enables edge clients to train collaboratively while preserving individual's data privacy. As clients do not inherently share identical data distributions, they may disagree in the direction of parameter updates, resulting in high compute and communication costs in comparison to centralized learning. Recent advances in FL focus on reducing data transmission during training; yet they neglected the increase of computational cost that dwarfs the merit of reduced communication. To this end, we propose FedDrop, which introduces channel-wise weighted dropout layers between convolutions to accelerate training while minimizing their impact on convergence. Empirical results show that FedDrop can drastically reduce the amount of FLOPs required for training with a small increase in communication, and push the Pareto frontier of communication/computation trade-off further than competing FL algorithms.
BibTeX
@inproceedings{liao2022feddrop,
author = {Dongping Liao and Xitong Gao and Yiren Zhao and Hao Dai and Li Li and Kafeng Wang and Kejiang Ye and Yang Wang and Cheng-zhong Xu},
title = {{FedDrop: Trajectory-weighted Dropout for Efficient Federated Learning}},
booktitle = {International Conference on Learning Representations},
year = {2022},
url = {https://openreview.net/forum?id=obi9EkyVeED},
}Loading related content...