Standard Federated Learning (FL) relies on a clean but often unrealistic assumption: while data is decentralized and private, it arrives neatly labeled on edge devices. In real-world deployments—ranging from clinical oncology archives (Alexander Chowdhury et al. in A Review of Medical Federated Learning: Applications in Oncology and Cancer Research) to on-device sensor tracking (Ying Han Pang et al. in A Review on Federated Learning on Sensor-Based Human Activity Recognition)—expert annotation is scarce, expensive, and often entirely absent at the edge. This creates a massive "label gap."
Federated Semi-Supervised Learning (FSSL) bridges this gap by combining decentralized orchestration with semi-supervised learning (SSL). It allows nodes with entirely unlabeled data to actively participate in training a global model, unlocking vast amounts of unannotated information stored across multiple distributed archives.
The Global Model as a Proxy Teacher
How do you train a model on data when the client doesn't know the ground truth? The core intuition of FSSL is to leverage the collective intelligence of the network as a "teacher." As shown by Xinyang Lin et al. in Federated Learning with Positive and Unlabeled Data, a global model can serve as a proxy teacher by providing pseudo-supervision based on the aggregated knowledge of the entire network. This mechanism allows an individual node to categorize its unlabeled samples into classes that may only be explicitly labeled on other devices in the federation.
This teacher-student dynamic manifests across various domains:
- Semantic Segmentation: In class-heterogeneous environments where clients only possess data for a subset of object classes, Jiaxu Miao et al. in FedSeg: Class-Heterogeneous Federated Learning for Semantic Segmentation demonstrate that the aggregated global model can provide pseudo-labels for pixels labeled as background in local datasets. This allows local clients to identify and learn from features belonging to classes they do not explicitly possess in their private data.
- Speech Recognition: On-device models can adapt to new domains without human-labeled transcripts. Haaris Mehmood et al. in FedNST: Federated Noisy Student Training for Automatic Speech Recognition utilize a pre-trained teacher model augmented with a language model and beam search to generate local pseudo-labels on private edge devices, bypassing manual transcription entirely.
- Graph Learning: In environments where data holders have limited labeled nodes, Bo Yan et al. in Federated Graph Condensation with Information Bottleneck Principles use a federated pre-training phase to generate pseudo-labels that guide subsequent feature extraction and graph condensation.
Anchoring to Prevent Confirmation Bias
A major risk in FSSL is confirmation bias: if a local model generates incorrect pseudo-labels, it can quickly reinforce its own errors and diverge, subsequently poisoning the global model during aggregation. To prevent this, several frameworks introduce server-led anchoring.
The authors of SemiFL: Semi-Supervised Federated Learning for Unlabeled Clients with Alternate Training (Enmao Diao et al.) propose an "alternate training" strategy. The pipeline alternates between server-side fine-tuning on a small labeled dataset and client-side training on unlabeled data with fixed pseudo-labels. This ensures the global model remains grounded in ground-truth reality while leveraging distributed unlabeled resources across multiple clients.
In resource-constrained IoT environments, Jiaqi Wang et al. in Knowledge-Enhanced Semi-Supervised Federated Learning for Aggregating Heterogeneous Lightweight Clients in IoT expand on this concept. They utilize a small labeled dataset and a large pretrained model at a central server to act as a stabilizing anchor. This configuration prevents local models from diverging during the pseudo-labeling process and ensures that decentralized training remains aligned with the target task. Similarly, in large language model applications, Ruhan Wang et al. in Federated In-Context Learning: Iterative Refinement for Improved Answer Quality show how clients can utilize a global context provided by a central server to iteratively refine and relabel their private local datasets.
Consistency and Consensus Regularization
Instead of relying purely on hard pseudo-labels from a single teacher, models can learn from unlabeled data by enforcing consistency across perturbations or building a consensus across multiple models.
Jie Ding et al. in Federated Learning Challenges and Opportunities: An Outlook highlight that consistency regularization techniques enable federated systems to utilize large volumes of unlabeled data by enforcing similar model outputs across perturbed versions of the same input. This method bridges the gap between devices with labeled data and those with only unlabeled observations, allowing for performance levels that approach centralized, fully-labeled training. This consistency-based objective is particularly effective for cross-border clinical data, as demonstrated by Dong Yang et al. in Federated Semi-Supervised Learning for COVID Region Segmentation in Chest CT using Multi-National Data from China, Italy, Japan, where consistency objectives on unsupervised clients improve generalization across highly heterogeneous data sources.
When data distributions are extremely complex or long-tailed—a common issue in real-world benchmarks like the cardiovascular data analyzed by Yukun Zhang et al. in FedCVD: The First Real-World Federated Learning Benchmark on Cardiovascular Disease Data—relying on a single model's weights can lead to severe model drift. To circumvent this, some frameworks share predictions rather than parameters:
- Prediction Consensus: Dongyang Fan et al. in Collaborative Learning via Prediction Consensus propose a decentralized method where agents update their local models by incorporating consensus pseudo-labels generated from the aggregated predictions of the collective on a common unlabeled dataset. A local disagreement loss aligns each local model with the group's collective expertise.
- Abstention-Aware Voting: Yukai Xu et al. in Privacy-Preserving Heterogeneous Federated Learning for Sensitive Healthcare Data utilize aggregated votes to generate pseudo-labels for a shared dataset, which are then distributed back to participants to refine local models without direct data or weight exchange.
Structural and Selective Assimilation
To further stabilize training, recent work integrates structural data relationships or allows clients to choose their source of truth. Jonathan Scott et al. in Cross-client Label Propagation for Transductive and Semi-Supervised Federated Learning integrate iterative graph-based label propagation into the federated training cycle. By continuously updating labels for unlabeled data using graph-derived relationships combined with model-centric updates, they significantly improve training stability.
Crucially, effective semi-supervised federated learning does not always require complex peer-to-peer communication or auxiliary server datasets. Yae Jee Cho et al. in Local or Global: Selective Knowledge Assimilation for Federated Learning with Limited Labels introduce a framework (FedLabel) where clients selectively assimilate knowledge from either their local model or the global server model. By leveraging these existing models as natural experts for pseudo-label generation, the system avoids the communication overhead typically associated with finding peer clients while achieving substantial accuracy gains.
Ultimately, as shown by Jay Mahadeokar et al. in Federated Domain Adaptation for ASR with Full Self-Supervision and Enmao Diao in Semi-Supervised Federated Learning for Keyword Spotting, bridging the label gap via local pseudo-label generation and robust loss functions allows decentralized systems to capture a substantial majority of the performance gains typically restricted to privacy-invasive, fully centralized training.
Key Foundational Work
A Review of Medical Federated Learning: Applications in Oncology and Cancer Research (2021) — contributed 1 concept elements to this direction.
Training Keyword Spotting Models on Non-IID Data with Federated Learning (2020) — contributed 1 concept elements to this direction.
Federated Semi-Supervised Learning for COVID Region Segmentation in Chest CT using Multi-National Data from China, Italy, Japan (2020) — contributed 1 concept elements to this direction.