Multi-Source Collaborative Gradient Discrepancy Minimization for Federated Domain Generalization

Yikang Wei, Yahong Han

Abstract

Federated Domain Generalization aims to learn a domain-invariant model from multiple decentralized source domains for deployment on unseen target domains. Due to privacy concerns, the data from different source domains are kept isolated, which poses challenges in bridging the domain gap. To address this issue, we propose a Multi-source Collaborative Gradient Discrepancy Minimization (MCGDM) method for federated domain generalization. Specifically, we propose intra-domain gradient matching between the original images and augmented images to avoid overfitting the domain-specific information within isolated domains. Additionally, we propose inter-domain gradient matching with the collaboration of other domains, which can further reduce the domain shift across decentralized domains. Combining intra-domain and inter-domain gradient matching, our method enables the learned model to generalize well on unseen domains. Furthermore, our method can be extended to the federated domain adaptation task by fine-tuning the target model on the pseudo-labeled target domain. The extensive experiments on federated domain generalization and adaptation indicate that our method outperforms the state-of-the-art methods significantly.

Introduction

In real-world applications, data is frequently siloed across different institutions or devices due to strict privacy regulations. While Federated Learning (FL) provides a framework for training models on decentralized data, it often fails when encountering Domain Shift—where the data distribution in the training sources differs significantly from the target environment.

Comparison between standard Domain Generalization (DG), Federated Domain Generalization (FedDG), and Federated Domain Adaptation (FedDA).

Federated Domain Generalization (FedDG) seeks to train a robust model across these isolated sources that can generalize to entirely unseen domains. Traditional approaches often attempt to share data "styles" or use generative models, but these can compromise privacy or fail to capture the complexity of real-world shifts. The challenge lies in aligning diverse domains without direct access to raw data or sensitive metadata.

Methodology: The MCGDM Framework

We introduce Multi-source Collaborative Gradient Discrepancy Minimization (MCGDM). The core philosophy is that if gradients from different domains (or different views of the same data) diverge, the model is likely learning domain-specific noise rather than universal semantic features. By minimizing this discrepancy, we force the model to converge on domain-invariant representations.

Overview of the MCGDM framework, detailing the intra-domain and inter-domain gradient matching processes.

1. Intra-domain Gradient Matching

On the local client side, the model calculates gradients for both original images and their augmented counterparts (using RandAugment). By minimizing the cosine distance between these gradients, the model is incentivized to focus on intrinsic features that remain constant despite visual transformations, effectively filtering out local domain noise.

2. Inter-domain Gradient Matching

To bridge the gap between isolated sources, the server distributes classifier heads from other domains to each local client. The client then ensures its local updates are consistent with the gradients produced by these "foreign" classifiers. These classifiers act as privacy-preserving anchors, allowing a client to verify if its learned features would be recognized by models trained on different data distributions.

Experimental Results

MCGDM was evaluated on several standard benchmarks, including PACS, VLCS, and Office-Home. The results demonstrate that MCGDM consistently outperforms state-of-the-art (SOTA) methods:

  • PACS: Achieved 86.2% accuracy, a 1.1% improvement over the previous best (FADH) and nearly 5% higher than standard FedAvg.
  • VLCS: Reached 78.1% accuracy, surpassing other FedDG methods by 2-3%.
  • Office-Home: Achieved 66.8%, demonstrating scalability to datasets with many categories (65).
  • Federated Domain Adaptation (FedDA): When extended to scenarios with unlabeled target data, MCGDM set new records on Digit-5 (97.1%) and Office-Caltech10 (98.6%).

Discussion and Key Insights

  • Gradients as Privacy-Preserving Proxies: Sharing classifier gradients provides a learning direction without exposing the underlying raw data distribution, offering a robust privacy layer.
  • Efficiency: To maintain practical communication costs, gradient matching is performed primarily on the classifier heads rather than the entire deep network, which is sufficient for learning invariant representations.
  • Limitations: The method's success is partially dependent on the quality of data augmentations. Furthermore, while sharing only classifier heads is efficient, the requirement to send multiple heads per round may pose challenges in extremely low-bandwidth environments.

In conclusion, MCGDM provides a powerful, privacy-conscious framework for training models that are truly robust to domain shifts in decentralized settings.

Summary updated

Abstract

Federated Domain Generalization aims to learn a domain-invariant model from multiple decentralized source domains for deployment on unseen target domain. Due to privacy concerns, the data from different source domains are kept isolated, which poses challenges in bridging the domain gap. To address this issue, we propose a Multi-source Collaborative Gradient Discrepancy Minimization (MCGDM) method for federated domain generalization. Specifically, we propose intra-domain gradient matching between the original images and augmented images to avoid overfitting the domain-specific information within isolated domains. Additionally, we propose inter-domain gradient matching with the collaboration of other domains, which can further reduce the domain shift across decentralized domains. Combining intra-domain and inter-domain gradient matching, our method enables the learned model to generalize well on unseen domains. Furthermore, our method can be extended to the federated domain adaptation task by fine-tuning the target model on the pseudo-labeled target domain. The extensive experiments on federated domain generalization and adaptation indicate that our method outperforms the state-of-the-art methods significantly.
BibTeX
@inproceedings{wei2024multi,
  author = {Yikang Wei and Yahong Han},
  title = {{Multi-Source Collaborative Gradient Discrepancy Minimization for Federated Domain Generalization}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i14.29510},
}
Figure