SemiDFL: A Semi-Supervised Paradigm for Decentralized Federated Learning

Xinyang Liu, Pengchao Han, Xuan Li, Bo Liu

TL;DR

SemiDFL is the first semi-supervised framework specifically designed for decentralized federated learning (DFL), where clients collaborate over a peer-to-peer network without any central server. By pairing peer-assisted pseudo-labeling with a decentralized, consensus-trained diffusion model for data augmentation, SemiDFL builds shared consensus in both the data and model spaces to achieve state-of-the-art accuracy under extreme label scarcity and non-IID data distributions.

What Problem Does This Paper Address?

Decentralized Federated Learning (DFL) removes the central parameter server found in standard federated learning, letting devices communicate directly with neighboring peers to avoid single-point failures and communication bottlenecks. However, existing DFL methods largely assume that every client has abundant labeled data.

In reality, data across edges is mostly unlabeled and heterogeneous: some clients have a few labeled examples, others have only unlabeled data, and some have a mix. Applying standard semi-supervised learning (SSL) techniques to DFL fails because:

  1. Severe local bias: In non-IID (non-identically and independently distributed) settings, a local client’s model cannot generate reliable pseudo-labels on its own.
  2. Data scarcity & divergence: Clients cannot share raw data due to privacy constraints, making data augmentation difficult.
  3. Lack of central validation: Without a server holding a proxy test set, peers cannot easily evaluate each other's models to weight peer aggregations appropriately.

Methods

SemiDFL addresses these issues by establishing consensus across two dimensions: data space and model space.

[ Unlabeled Data ] 
       │
       ▼
1. Neighborhood Pseudo-Labeling (NPL)
   (Ensemble predictions across neighbor models + dynamic filtering threshold)
       │
       ▼
2. Consensus Diffusion Training & MixUp (C-MixUp)
   (Train local diffusion model → exchange with peers → generate shared distribution data)
       │
       ▼
3. Local Classifier Training
   (Train classifier on: Labeled ∪ Pseudo-Labeled ∪ Diffusion-Generated Data)
       │
       ▼
4. Performance-Weighted Adaptive Aggregation
   (Evaluate neighbor classifiers on generated data to dynamically weight peer consensus)

The system relies on three core technical components:

  • Neighborhood Pseudo-Labeling (NPL): Instead of relying only on its own local classifier, each client randomly queries its neighbors' classifiers across augmented variants of unlabeled samples. It also dynamically sets class-wise confidence thresholds based on the maximum qualified pseudo-label counts found within its local neighborhood sub-graph.
  • Consensus MixUp (C-MixUp): Every client trains a local conditional diffusion model on its available labeled/pseudo-labeled data. Diffusion model weights are aggregated across neighbors via decentralized consensus. As these generative models converge to a shared data distribution, clients sample synthetic data and perform MixUp interpolation alongside local real data, overcoming data islands without sharing raw records.
  • Adaptive Aggregation (AdaGen): To weight incoming model updates without a shared central validation set, each client draws a small synthetic validation sample ($~100$ items) from its local diffusion model. Neighbors that achieve higher accuracy on this synthetic validation set receive exponentially higher weights in the consensus update.

Datasets

The authors evaluated SemiDFL against standard vision benchmarks:

  • MNIST (tested with a Convolutional Neural Network / CNN)
  • Fashion-MNIST (tested with CNN)
  • CIFAR-10 (tested with ResNet-18)

Data was distributed across heterogeneous clients using Dirichlet distribution simulations controlled by $\alpha$ (where lower $\alpha$, such as $\alpha = 0.1$, indicates severe non-IID label skew). Labeled data fractions ($r$) were evaluated down to extreme low-label regimes ($r = 0.1\%$ to $5\%$).

Key Insights and Ideas

  • Decentralized Generative Models as Virtual Data Bridges: In DFL, raw data cannot leave the device. However, diffusing model weights across peers allows clients to collaboratively learn the underlying global distribution. Sampling from this shared generative model effectively synthesizes a "proxy dataset" that acts as a bridge across non-IID splits.
  • Zero-Leakage Model Evaluation: A recurring challenge in decentralized consensus is evaluating peer reliability. SemiDFL shows that synthetic images sampled from the consensus diffusion model can substitute for a global validation set (AdaGen), performing competitively with an idealized oracle test set (AdaTest) while preserving client privacy.
  • Neighborhood Ensembling Stabilizes Noisy Labels: Standard pseudo-labeling diverges rapidly under client drift. Combining stochastic neighbor predictions stabilizes pseudo-labels much like an ensemble model, preventing early confirmation bias.

Key Results

SemiDFL consistently outperformed adapted baselines (MixMatch, FlexMatch, and CBAFed) across all datasets, especially under high non-IID skew ($\alpha = 0.1$) and minimal supervision:

  • CIFAR-10 ($\alpha = 0.1, r = 1\%$ labeled data): SemiDFL reached $40.28 \pm 0.89\%$ accuracy, while FlexMatch and CBAFed failed to converge or dropped substantially ($21.50\%$ for CBAFed; MixMatch hit $30.53\%$).
  • CIFAR-10 ($\alpha = 100, r = 5\%$ labeled data): SemiDFL achieved $69.40 \pm 0.20\%$, significantly surpassing FlexMatch ($56.90\%$) and MixMatch ($65.54\%$).
  • Ablations:
  • Replacing standard pseudo-labeling with NPL boosted CIFAR-10 accuracy from $40.51\%$ to $69.40\%$ ($\alpha=100, r=5\%$).
  • Using a consensus diffusion model outperformed GAN-based generation by $\sim 1.5\text{--}2.3\%$ on MNIST and outperformed local-only MixUp by nearly $20\%$ in low-data regimes.

Limitations and Future Work

  • Computation and Communication Overhead: Training both a task classifier and a denoising diffusion model locally—and transmitting both sets of parameters between peers every round—adds substantial compute and bandwidth demands, which could strain resource-constrained edge devices.
  • Task Complexity: Experiments were limited to basic benchmark datasets (MNIST, Fashion-MNIST, CIFAR-10) with relatively simple network architectures. Evaluating larger generative models on higher-resolution, open-world datasets remains open.
  • Communication Topologies: The paper validates performance on a fixed 10-node decentralized topology. The impact of high-latency networks, dynamic dropouts, or time-varying ad-hoc graphs was not explored.

Summary updated

Abstract

Decentralized federated learning (DFL) realizes cooperative model training among connected clients without relying on a central server, thereby mitigating communication bottlenecks and eliminating the single-point failure issue present in centralized federated learning (CFL). Most existing work on DFL focuses on supervised learning, assuming each client possesses sufficient labeled data for local training. However, in real-world applications, much of the data is unlabeled. We address this by considering a challenging yet practical semi-supervised learning (SSL) scenario in DFL, where clients may have varying data sources: some with few labeled samples, some with purely unlabeled data, and others with both. In this work, we propose SemiDFL, the first semi-supervised DFL method that enhances DFL performance in SSL scenarios by establishing a consensus in both data and model spaces. Specifically, we utilize neighborhood information to improve the quality of pseudo-labeling, which is crucial for effectively leveraging unlabelled data. We then design a consensus-based diffusion model to generate synthesized data, which is used in combination with pseudo-labeled data to create mixed datasets. Additionally, we develop an adaptive aggregation method that leverages the model accuracy of synthesized data to further enhance SemiDFL performance. Through extensive experimentation, we demonstrate the remarkable performance superiority of the proposed DFL-Semi method over existing CFL and DFL schemes in both iid and Non-iid SSL scenarios.
BibTeX
@inproceedings{liu2025semidfl,
  author = {Xinyang Liu and Pengchao Han and Xuan Li and Bo Liu},
  title = {{SemiDFL: A Semi-Supervised Paradigm for Decentralized Federated Learning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  url = {https://openreview.net/forum?id=Ohazg1lQZU},
}

Loading related content...

Figure