TL;DR
SemiDFL is the first semi-supervised framework specifically designed for decentralized federated learning (DFL), where clients collaborate over a peer-to-peer network without any central server. By pairing peer-assisted pseudo-labeling with a decentralized, consensus-trained diffusion model for data augmentation, SemiDFL builds shared consensus in both the data and model spaces to achieve state-of-the-art accuracy under extreme label scarcity and non-IID data distributions.
What Problem Does This Paper Address?
Decentralized Federated Learning (DFL) removes the central parameter server found in standard federated learning, letting devices communicate directly with neighboring peers to avoid single-point failures and communication bottlenecks. However, existing DFL methods largely assume that every client has abundant labeled data.
In reality, data across edges is mostly unlabeled and heterogeneous: some clients have a few labeled examples, others have only unlabeled data, and some have a mix. Applying standard semi-supervised learning (SSL) techniques to DFL fails because:
- Severe local bias: In non-IID (non-identically and independently distributed) settings, a local client’s model cannot generate reliable pseudo-labels on its own.
- Data scarcity & divergence: Clients cannot share raw data due to privacy constraints, making data augmentation difficult.
- Lack of central validation: Without a server holding a proxy test set, peers cannot easily evaluate each other's models to weight peer aggregations appropriately.
Methods
SemiDFL addresses these issues by establishing consensus across two dimensions: data space and model space.
[ Unlabeled Data ]
│
▼
1. Neighborhood Pseudo-Labeling (NPL)
(Ensemble predictions across neighbor models + dynamic filtering threshold)
│
▼
2. Consensus Diffusion Training & MixUp (C-MixUp)
(Train local diffusion model → exchange with peers → generate shared distribution data)
│
▼
3. Local Classifier Training
(Train classifier on: Labeled ∪ Pseudo-Labeled ∪ Diffusion-Generated Data)
│
▼
4. Performance-Weighted Adaptive Aggregation
(Evaluate neighbor classifiers on generated data to dynamically weight peer consensus)
The system relies on three core technical components:
- Neighborhood Pseudo-Labeling (NPL): Instead of relying only on its own local classifier, each client randomly queries its neighbors' classifiers across augmented variants of unlabeled samples. It also dynamically sets class-wise confidence thresholds based on the maximum qualified pseudo-label counts found within its local neighborhood sub-graph.
- Consensus MixUp (C-MixUp): Every client trains a local conditional diffusion model on its available labeled/pseudo-labeled data. Diffusion model weights are aggregated across neighbors via decentralized consensus. As these generative models converge to a shared data distribution, clients sample synthetic data and perform MixUp interpolation alongside local real data, overcoming data islands without sharing raw records.
- Adaptive Aggregation (AdaGen): To weight incoming model updates without a shared central validation set, each client draws a small synthetic validation sample ($~100$ items) from its local diffusion model. Neighbors that achieve higher accuracy on this synthetic validation set receive exponentially higher weights in the consensus update.
Datasets
The authors evaluated SemiDFL against standard vision benchmarks:
- MNIST (tested with a Convolutional Neural Network / CNN)
- Fashion-MNIST (tested with CNN)
- CIFAR-10 (tested with ResNet-18)
Data was distributed across heterogeneous clients using Dirichlet distribution simulations controlled by $\alpha$ (where lower $\alpha$, such as $\alpha = 0.1$, indicates severe non-IID label skew). Labeled data fractions ($r$) were evaluated down to extreme low-label regimes ($r = 0.1\%$ to $5\%$).
Key Insights and Ideas
- Decentralized Generative Models as Virtual Data Bridges: In DFL, raw data cannot leave the device. However, diffusing model weights across peers allows clients to collaboratively learn the underlying global distribution. Sampling from this shared generative model effectively synthesizes a "proxy dataset" that acts as a bridge across non-IID splits.
- Zero-Leakage Model Evaluation: A recurring challenge in decentralized consensus is evaluating peer reliability. SemiDFL shows that synthetic images sampled from the consensus diffusion model can substitute for a global validation set (
AdaGen), performing competitively with an idealized oracle test set (AdaTest) while preserving client privacy. - Neighborhood Ensembling Stabilizes Noisy Labels: Standard pseudo-labeling diverges rapidly under client drift. Combining stochastic neighbor predictions stabilizes pseudo-labels much like an ensemble model, preventing early confirmation bias.
Key Results
SemiDFL consistently outperformed adapted baselines (MixMatch, FlexMatch, and CBAFed) across all datasets, especially under high non-IID skew ($\alpha = 0.1$) and minimal supervision:
- CIFAR-10 ($\alpha = 0.1, r = 1\%$ labeled data): SemiDFL reached $40.28 \pm 0.89\%$ accuracy, while FlexMatch and CBAFed failed to converge or dropped substantially ($21.50\%$ for CBAFed; MixMatch hit $30.53\%$).
- CIFAR-10 ($\alpha = 100, r = 5\%$ labeled data): SemiDFL achieved $69.40 \pm 0.20\%$, significantly surpassing FlexMatch ($56.90\%$) and MixMatch ($65.54\%$).
- Ablations:
- Replacing standard pseudo-labeling with NPL boosted CIFAR-10 accuracy from $40.51\%$ to $69.40\%$ ($\alpha=100, r=5\%$).
- Using a consensus diffusion model outperformed GAN-based generation by $\sim 1.5\text{--}2.3\%$ on MNIST and outperformed local-only MixUp by nearly $20\%$ in low-data regimes.
Limitations and Future Work
- Computation and Communication Overhead: Training both a task classifier and a denoising diffusion model locally—and transmitting both sets of parameters between peers every round—adds substantial compute and bandwidth demands, which could strain resource-constrained edge devices.
- Task Complexity: Experiments were limited to basic benchmark datasets (MNIST, Fashion-MNIST, CIFAR-10) with relatively simple network architectures. Evaluating larger generative models on higher-resolution, open-world datasets remains open.
- Communication Topologies: The paper validates performance on a fixed 10-node decentralized topology. The impact of high-latency networks, dynamic dropouts, or time-varying ad-hoc graphs was not explored.
Summary updated