SemiDFL: A Semi-Supervised Paradigm for Decentralized Federated Learning

Xinyang Liu, Pengchao Han, Xuan Li, Bo Liu

Abstract

Decentralized federated learning (DFL) realizes cooperative model training among connected clients without relying on a central server, thereby mitigating communication bottlenecks and eliminating the single-point failure issue present in centralized federated learning (CFL). Most existing work on DFL focuses on supervised learning, assuming each client possesses sufficient labeled data for local training. However, in real-world applications, much of the data is unlabeled. We address this by considering a challenging yet practical semisupervised learning (SSL) scenario in DFL, where clients may have varying data sources: some with few labeled samples, some with purely unlabeled data, and others with both. In this work, we propose SemiDFL, the first semi-supervised DFL method that enhances DFL performance in SSL scenarios by establishing a consensus in both data and model spaces. Specifically, we utilize neighborhood information to improve the quality of pseudo-labeling, which is crucial for effectively leveraging unlabeled data. We then design a consensusbased diffusion model to generate synthesized data, which is used in combination with pseudo-labeled data to create mixed datasets. Additionally, we develop an adaptive aggregation method that leverages the model accuracy of synthesized data to further enhance SemiDFL performance. Through extensive experimentation, we demonstrate the remarkable performance superiority of the proposed DFL-Semi method over existing CFL and DFL schemes in both IID and non-IID SSL scenarios.

Introduction

Federated Learning (FL) has emerged as a key paradigm for privacy-preserving machine learning. While Centralized Federated Learning (CFL) relies on a central server, Decentralized Federated Learning (DFL) allows clients to communicate directly with neighbors, enhancing scalability and robustness. However, DFL typically assumes a supervised setting with abundant labeled data. In practice, data is often unlabeled and distributed heterogeneously (non-IID) across the network. SemiDFL addresses this gap by introducing a framework for semi-supervised learning in decentralized environments.

Methodology: The SemiDFL Framework

SemiDFL establishes consensus in both model and data spaces through a six-step cyclic process designed to overcome the limitations of local, biased models in label-scarce environments.

Figure 1: Schematic diagram of the SemiDFL framework, illustrating the six-step cycle from neighborhood pseudo-labeling to adaptive aggregation.

The framework is built on three main pillars:

  1. Neighborhood Pseudo-Labeling (NPL): Clients improve pseudo-label quality by consulting a "committee" of neighboring classifiers rather than relying on a single biased local model. An adaptive threshold filters low-confidence predictions to mitigate the impact of non-IID data.
  2. Consensus MixUp (C-MixUp): A diffusion model is trained across the network via consensus to generate synthetic data representing the global distribution. This synthetic data is used for MixUp augmentation, bridging "data islands" without sharing raw private data.
  3. Adaptive Aggregation: Synthetic data serves as a virtual validation set. Clients evaluate neighbors' models on this consensus data, assigning higher weights to better-performing models during aggregation, creating a more effective consensus model space.

Experimental Results

The researchers evaluated SemiDFL on MNIST, Fashion-MNIST, and CIFAR-10 datasets, simulating various non-IID degrees using a Dirichlet distribution.

Figure 2: Accuracy vs. Non-IID Degree alpha, showing SemiDFL's superior stability compared to baselines as data heterogeneity increases.

Key findings include:

  • Superior Accuracy: On CIFAR-10 with only 5% labeled data, SemiDFL achieved 69.40% accuracy, significantly outperforming FlexMatch (56.90%) and MixMatch (65.54%) in the same decentralized setting.
  • Robustness to Heterogeneity: As shown in Figure 2, SemiDFL maintains high performance even in highly skewed non-IID scenarios where traditional methods' accuracies plummet.
  • Component Efficacy: Ablation studies confirmed that NPL, C-MixUp, and Adaptive Aggregation each contribute significantly to overall performance, with the diffusion-based C-MixUp providing a particularly large boost in non-IID scenarios.

Conclusion

SemiDFL provides a robust solution for semi-supervised decentralized learning by leveraging neighborhood consensus and synthetic data generation. While computational and communication overheads from diffusion models remain a challenge for resource-constrained edge devices, SemiDFL sets a new benchmark for accuracy and stability in heterogeneous, label-scarce decentralized environments.

Summary updated

Abstract

Decentralized federated learning (DFL) realizes cooperative model training among connected clients without relying on a central server, thereby mitigating communication bottlenecks and eliminating the single-point failure issue present in centralized federated learning (CFL). Most existing work on DFL focuses on supervised learning, assuming each client possesses sufficient labeled data for local training. However, in real-world applications, much of the data is unlabeled. We address this by considering a challenging yet practical semisupervised learning (SSL) scenario in DFL, where clients may have varying data sources: some with few labeled samples, some with purely unlabeled data, and others with both. In this work, we propose SemiDFL, the first semi-supervised DFL method that enhances DFL performance in SSL scenarios by establishing a consensus in both data and model spaces. Specifically, we utilize neighborhood information to improve the quality of pseudo-labeling, which is crucial for effectively leveraging unlabeled data. We then design a consensusbased diffusion model to generate synthesized data, which is used in combination with pseudo-labeled data to create mixed datasets. Additionally, we develop an adaptive aggregation method that leverages the model accuracy of synthesized data to further enhance SemiDFL performance. Through extensive experimentation, we demonstrate the remarkable performance superiority of the proposed DFL-Semi method over existing CFL and DFL schemes in both IID and non-IID SSL scenarios.
BibTeX
@inproceedings{liu2025semidfl,
  author = {Xinyang Liu and Pengchao Han and Xuan Li and Bo Liu},
  title = {{SemiDFL: A Semi-Supervised Paradigm for Decentralized Federated Learning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  url = {https://openreview.net/forum?id=Ohazg1lQZU},
}
Figure