FedUDG: Federated Domain Generalization with Unlabeled Clients

Junyu Ren, Wensheng Gan, Philip S. YuarXiv 2026 · arXiv

TL;DR

FedUDG is a novel federated learning framework that enables models to generalize to unseen, out-of-distribution (OOD) environments even when most participating clients lack labeled data. By strategically separating core class semantics from domain-specific noise through "semantic anchors," it achieves state-of-the-art performance on OOD tasks while providing a rigorous differentially private extension.

What Problem Does This Paper Address?

In the real world, data is messy and labels are expensive. Federated Learning (FL) allows multiple parties to train a model without sharing raw data, but it faces two major hurdles:

  1. Domain Shift: A model trained on "Photos" might fail when deployed on "Sketches" (Out-of-Distribution).
  2. Label Scarcity: Most existing Federated Domain Generalization (FDG) methods assume every client has perfectly labeled data. In reality, a few "rich" clients might have labels, while many "poor" clients have only unlabeled raw data.

Existing semi-supervised methods (FSSL) often fail here because they tend to overfit to domain-specific "shortcuts" (e.g., a specific background color) to improve local accuracy, which actually harms their ability to generalize to new environments. FedUDG aims to bridge this gap, as illustrated in the problem landscape below:

Fig. 1. Comparison of the problem settings of FSSL, FDG, and this work. FSSL jointly exploits labeled and unlabeled data but typically evaluates on the client distributions seen during training. FDG targets OOD generalization to unseen target clients but generally assumes that all source clients are labeled. FedUDG addresses both heterogeneous label availability across source clients and generalization to an unseen OOD target client.

Methods

The authors introduce a "parameter-functional differentiation" strategy. The core idea is to prevent unlabeled data from corrupting the fundamental "meaning" of a class.

  • Semantic Anchors: The server maintains a set of "anchors" (class prototypes) that are updated only by clients with ground-truth labels. This serves as a stable, "pure" reference for what a "dog" or "car" actually looks like across domains.
  • Student-Teacher Framework: Unlabeled clients use an Exponential Moving Average (EMA) teacher model to generate targets.
  • Credible Self-Training: To avoid learning from noisy pseudo-labels, a sample is only used for training if the biased classifier and the "pure" semantic anchor agree on the class with high confidence.
  • Label-free Distillation: For all unlabeled samples (even those that fail the confidence test), the model performs representation distillation. This encourages the student to produce features consistent with the teacher, stabilizing the model against appearance variations (like lighting or style) without needing a class label.
  • DP-FedUDG: The authors also propose a version using Differential Privacy (DP-SGD) to provide formal privacy guarantees, quantifying the inevitable trade-off between privacy and accuracy.

The overall architecture is detailed in Figure 2:

Fig. 2. Overall architecture of FedUDG. Labeled clients learn the encoder, classifier, and semantic anchors from ground-truth labels. Within a student-teacher framework, unlabeled clients perform semantic self-training on credible samples and label-free representation distillation on all samples. The server follows the parameter-functional differentiation strategy when aggregating model components and updates the EMA teacher.

Datasets

The framework was evaluated on three major visual benchmarks:

  • PACS: (Photo, Art Painting, Cartoon, Sketch) - 7 classes.
  • miniDomainNet: A 7-class subset of the massive DomainNet dataset.
  • VLCS: (VOC2007, LabelMe, Caltech101, SUN09) - 5 classes.

The researchers tested four distinct protocols, varying the number of labeled vs. unlabeled clients and introducing significant class imbalances to simulate realistic federated environments.

Key Insights and Ideas

  • Separation of Concerns: By isolating class semantics (anchors) from general feature learning, the model can "absorb" knowledge from unlabeled domains without letting domain-specific noise erode the class definitions.
  • Consensus as a Filter: Requiring agreement between a biased classifier and a stable semantic anchor is a much more robust way to filter pseudo-labels than simple confidence thresholds.
  • Unlabeled Data is a Double-Edged Sword: The results show that simply adding unlabeled data using standard semi-supervised methods can actually decrease OOD performance. FedUDG proves that unlabeled data can help, but only if you explicitly constrain the influence of domain-specific information.

Key Results

FedUDG consistently outperformed both FDG baselines (which ignore unlabeled data) and FSSL baselines (which use it but don't target generalization).

  • Superior Accuracy: FedUDG achieved the highest mean accuracy in 12 out of 13 evaluated settings. On the PACS-DomainNet joint dataset, it outperformed the strongest FDG baseline by up to 3.5% and the strongest FSSL baseline by over 10%.
  • Stability: As the number of unlabeled clients increased, FedUDG's performance remained stable, while baselines often degraded.
  • Better Feature Alignment: t-SNE visualizations (see Figure 3 in the paper) qualitatively show that FedUDG forms much tighter, better-aligned clusters of classes across different domains compared to other methods.

The training configuration used for these results is summarized below:

Model and Training Configuration

Limitations and Future Work

  • The Privacy Tax: While DP-FedUDG provides formal guarantees, the "privacy tax" is high. Accuracy dropped significantly (e.g., from ~86% to ~57% at a privacy budget of $\epsilon \approx 17$). Finding ways to preserve OOD utility under strict privacy constraints remains a major challenge.
  • Synergy with FDG: Currently, FedUDG uses standard training for its labeled clients. A promising future direction is integrating advanced FDG techniques (like adversarial alignment) specifically for the labeled subset to further boost the "anchor" quality.
  • Hyperparameter Sensitivity: While the authors show FedUDG isn't overly sensitive to its new parameters ($\tau, \kappa, \alpha$), finding the optimal balance still requires some tuning based on the specific data distribution.

Summary updated

Abstract

Federated domain generalization (FDG) aims to learn models that generalize reliably to out-of-distribution (OOD) environments without centralizing client's data. However, existing FDG methods assume that all source clients have labeled data, limiting their ability to leverage the abundant unlabeled client data available in practice. To address this limitation, we propose FedUDG. At the federation level, FedUDG utilizes a parameter-functional differentiation strategy that maintains a semantic anchor updated solely using ground-truth labels. The remaining parameters absorb knowledge from unlabeled domains under bounded aggregation, preventing domain-biased noise from eroding the class semantics. Within each unlabeled client, FedUDG utilizes a student-teacher framework with two complementary objectives: (1) semantic self-training on high-confidence samples, using the functional differentiation between the semantic anchor and classifier to constrain the model to learn semantically pure and cross-domain stable features; and (2) label-free representation distillation on all unlabeled samples to stabilize learning under appearance variations. Together, these objectives guide the model toward a transferable feature space, while mitigating overfitting to domain-specific characteristics. Furthermore, to address the lack of systematic evaluation of FDG under formal privacy constraints, we develop a sample-level differentially private extension, DP-FedUDG, to quantify the trade-off between privacy and OOD generalization. Across four evaluation protocols, FedUDG consistently outperforms state-of-the-art baselines. The code is available at https://github.com/Jr10426/FedUDG
BibTeX
@misc{ren2026fedudg,
  author = {Junyu Ren and Wensheng Gan and Philip S. Yu},
  title = {{FedUDG: Federated Domain Generalization with Unlabeled Clients}},
  year = {2026},
  eprint = {2611.01498},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.01498},
}
Figure