Handling Covariate Shifts in Federated Learning with Generalization Guarantees
Lists
Only you see this
Abstract
Covariate shift across clients is a major challenge for federated learning (FL). This work studies the generalization properties of FL under intra-client and inter-client covariate shifts. To this end, we propose Federated Importance-weighteD Empirical risk Minimization (FIDEM) to optimize a global FL model, along with new variants of density ratio matching methods, aiming to handle covariate shifts. These methods trade off some level of privacy for improving the overall generalization performance. We theoretically show that FIDEM achieves smaller generalization error than classical empirical risk minimization under some certain settings. Experimental results demonstrate the superiority of FIDEM over federated averaging (McMahan et al., 2017) and other baselines, which would open the door to study FL under distribution shifts more systematically.
BibTeX
@inproceedings{ramezanikebrya2023handling,
author = {Ali Ramezani-Kebrya and Fanghui Liu and Thomas Pethick and Grigorios Chrysos and Volkan Cevher},
title = {{Handling Covariate Shifts in Federated Learning with Generalization Guarantees}},
booktitle = {International Conference on Learning Representations},
year = {2023},
url = {https://openreview.net/forum?id=Ho7W1yr8tV},
}