Federated Semi-Supervised Learning

37 concepts28 papers1 proposal

Standard Federated Learning (FL) relies on a clean but often unrealistic assumption: while data is decentralized and private, it arrives neatly labeled on edge devices. In real-world deployments—ranging from clinical oncology archives (Alexander Chowdhury et al. in A Review of Medical Federated Learning: Applications in Oncology and Cancer Research) to on-device sensor tracking (Ying Han Pang et al. in A Review on Federated Learning on Sensor-Based Human Activity Recognition)—expert annotation is scarce, expensive, and often entirely absent at the edge. This creates a massive "label gap."

Federated Semi-Supervised Learning (FSSL) bridges this gap by combining decentralized orchestration with semi-supervised learning (SSL). It allows nodes with entirely unlabeled data to actively participate in training a global model, unlocking vast amounts of unannotated information stored across multiple distributed archives.

The Global Model as a Proxy Teacher

How do you train a model on data when the client doesn't know the ground truth? The core intuition of FSSL is to leverage the collective intelligence of the network as a "teacher." As shown by Xinyang Lin et al. in Federated Learning with Positive and Unlabeled Data, a global model can serve as a proxy teacher by providing pseudo-supervision based on the aggregated knowledge of the entire network. This mechanism allows an individual node to categorize its unlabeled samples into classes that may only be explicitly labeled on other devices in the federation.

This teacher-student dynamic manifests across various domains:

Anchoring to Prevent Confirmation Bias

A major risk in FSSL is confirmation bias: if a local model generates incorrect pseudo-labels, it can quickly reinforce its own errors and diverge, subsequently poisoning the global model during aggregation. To prevent this, several frameworks introduce server-led anchoring.

The authors of SemiFL: Semi-Supervised Federated Learning for Unlabeled Clients with Alternate Training (Enmao Diao et al.) propose an "alternate training" strategy. The pipeline alternates between server-side fine-tuning on a small labeled dataset and client-side training on unlabeled data with fixed pseudo-labels. This ensures the global model remains grounded in ground-truth reality while leveraging distributed unlabeled resources across multiple clients.

In resource-constrained IoT environments, Jiaqi Wang et al. in Knowledge-Enhanced Semi-Supervised Federated Learning for Aggregating Heterogeneous Lightweight Clients in IoT expand on this concept. They utilize a small labeled dataset and a large pretrained model at a central server to act as a stabilizing anchor. This configuration prevents local models from diverging during the pseudo-labeling process and ensures that decentralized training remains aligned with the target task. Similarly, in large language model applications, Ruhan Wang et al. in Federated In-Context Learning: Iterative Refinement for Improved Answer Quality show how clients can utilize a global context provided by a central server to iteratively refine and relabel their private local datasets.

Consistency and Consensus Regularization

Instead of relying purely on hard pseudo-labels from a single teacher, models can learn from unlabeled data by enforcing consistency across perturbations or building a consensus across multiple models.

Jie Ding et al. in Federated Learning Challenges and Opportunities: An Outlook highlight that consistency regularization techniques enable federated systems to utilize large volumes of unlabeled data by enforcing similar model outputs across perturbed versions of the same input. This method bridges the gap between devices with labeled data and those with only unlabeled observations, allowing for performance levels that approach centralized, fully-labeled training. This consistency-based objective is particularly effective for cross-border clinical data, as demonstrated by Dong Yang et al. in Federated Semi-Supervised Learning for COVID Region Segmentation in Chest CT using Multi-National Data from China, Italy, Japan, where consistency objectives on unsupervised clients improve generalization across highly heterogeneous data sources.

When data distributions are extremely complex or long-tailed—a common issue in real-world benchmarks like the cardiovascular data analyzed by Yukun Zhang et al. in FedCVD: The First Real-World Federated Learning Benchmark on Cardiovascular Disease Data—relying on a single model's weights can lead to severe model drift. To circumvent this, some frameworks share predictions rather than parameters:

Structural and Selective Assimilation

To further stabilize training, recent work integrates structural data relationships or allows clients to choose their source of truth. Jonathan Scott et al. in Cross-client Label Propagation for Transductive and Semi-Supervised Federated Learning integrate iterative graph-based label propagation into the federated training cycle. By continuously updating labels for unlabeled data using graph-derived relationships combined with model-centric updates, they significantly improve training stability.

Crucially, effective semi-supervised federated learning does not always require complex peer-to-peer communication or auxiliary server datasets. Yae Jee Cho et al. in Local or Global: Selective Knowledge Assimilation for Federated Learning with Limited Labels introduce a framework (FedLabel) where clients selectively assimilate knowledge from either their local model or the global server model. By leveraging these existing models as natural experts for pseudo-label generation, the system avoids the communication overhead typically associated with finding peer clients while achieving substantial accuracy gains.

Ultimately, as shown by Jay Mahadeokar et al. in Federated Domain Adaptation for ASR with Full Self-Supervision and Enmao Diao in Semi-Supervised Federated Learning for Keyword Spotting, bridging the label gap via local pseudo-label generation and robust loss functions allows decentralized systems to capture a substantial majority of the performance gains typically restricted to privacy-invasive, fully centralized training.

Key Foundational Work

A Review of Medical Federated Learning: Applications in Oncology and Cancer Research (2021) — contributed 1 concept elements to this direction.
Training Keyword Spotting Models on Non-IID Data with Federated Learning (2020) — contributed 1 concept elements to this direction.
Federated Semi-Supervised Learning for COVID Region Segmentation in Chest CT using Multi-National Data from China, Italy, Japan (2020) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_decentralized_semi_supervised_learning_b6585df7,
  title = {decentralized semi-supervised learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/decentralized-semi-supervised-learning/rdr_b6585df74c5a4524ad8842ade45db1fe},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_decentralized_semi_supervised_learning_b6585df7_v1,
  title = {decentralized semi-supervised learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/decentralized-semi-supervised-learning/rdr_b6585df74c5a4524ad8842ade45db1fe/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake