Handling Covariate Shifts in Federated Learning with Generalization Guarantees

Ali Ramezani-Kebrya, Fanghui Liu, Thomas Pethick, Grigorios Chrysos, Volkan CevherICLR 2023 · OpenReview

Abstract

Covariate shift across clients is a major challenge for federated learning (FL). This work studies the generalization properties of FL under intra-client and inter-client covariate shifts. To this end, we propose Federated Importance-weighteD Empirical risk Minimization (FIDEM) to optimize a global FL model, along with new variants of density ratio matching methods, aiming to handle covariate shifts. These methods trade off some level of privacy for improving the overall generalization performance. We theoretically show that FIDEM achieves smaller generalization error than classical empirical risk minimization under some certain settings. Experimental results demonstrate the superiority of FIDEM over federated averaging (McMahan et al., 2017) and other baselines, which would open the door to study FL under distribution shifts more systematically.
BibTeX
@inproceedings{ramezanikebrya2023handling,
  author = {Ali Ramezani-Kebrya and Fanghui Liu and Thomas Pethick and Grigorios Chrysos and Volkan Cevher},
  title = {{Handling Covariate Shifts in Federated Learning with Generalization Guarantees}},
  booktitle = {International Conference on Learning Representations},
  year = {2023},
  url = {https://openreview.net/forum?id=Ho7W1yr8tV},
}
Figure