DFCA: Decentralized Federated Clustering Algorithm

Jonas Kirch, Sebastian Becker, Tiago Koketsu Rodrigues, Stefan Harmeling

Introduction

Standard Federated Learning (FL) architectures typically rely on a central server to coordinate model training across a network of clients. While effective, this centralized approach introduces a single point of failure and significant communication bottlenecks, particularly in Internet of Things (IoT) environments characterized by intermittent connectivity and limited bandwidth. Furthermore, real-world data is rarely Independent and Identically Distributed (IID). A single global model often fails to capture the nuances of heterogeneous data distributions across different device groups.

Clustered Federated Learning (CFL) addresses this by grouping similar clients to train specialized cluster models. However, existing state-of-the-art CFL methods, such as IFCA, still depend on a central coordinator. The Decentralized Federated Clustering Algorithm (DFCA) proposes a fully serverless, peer-to-peer alternative that maintains the benefits of clustering without the drawbacks of centralization.

Methodology: The DFCA Framework

DFCA enables clients to communicate exclusively with their immediate neighbors in a graph topology. The algorithm operates through a three-step iterative loop designed for decentralized environments.

The three-step iterative loop of the DFCA algorithm: (1) Cluster Assignment, (2) Local Training, and (3) Parameter Exchange.

1. Cluster Assignment

At the start of each round, a client evaluates all $k$ available cluster models against its local dataset. It performs local inference and self-assigns to the cluster model that yields the lowest empirical loss. This ensures that each device contributes to the model that best represents its specific data distribution.

2. Local Update

Once assigned, the client performs Stochastic Gradient Descent (SGD) to refine the chosen cluster model using its local data. Only the parameters of the assigned model are updated, preserving the specialization of the other $k-1$ models.

3. Decentralized Aggregation

The core innovation of DFCA lies in its "Sequential Aggregation" mechanism. Unlike traditional batch aggregation, which requires waiting for all neighbors to complete their updates, DFCA integrates neighbor updates as they arrive. By maintaining a sequential running average, the algorithm supports asynchronous communication and significantly reduces memory overhead. This makes it highly resilient to the "straggler" problem common in IoT networks.

The researchers evaluated two initialization strategies:

  • DFCA-GI (Global Initialization): All clients begin with identical weights for all cluster models.
  • DFCA-LI (Local Initialization): Clients start with personalized weights, which can enhance stability in highly noisy environments.

Experimental Evaluation

The performance of DFCA was benchmarked using several computer vision datasets modified to simulate real-world heterogeneity, including MNIST, EMNIST, FEMNIST, and CIFAR-10. For the MNIST and EMNIST datasets, digits were rotated (0°, 90°, 180°, 270°) to create synthetic clusters representing distinct data distributions.

Key Results

Experiments demonstrated that DFCA achieves performance parity with centralized state-of-the-art methods. On the EMNIST dataset with four clusters, DFCA-GI reached an accuracy of 85.7%, matching the centralized IFCA baseline. In contrast, other decentralized methods like FedSPD and D-FedAvg-M lagged behind, achieving significantly lower accuracy.

Comprehensive experimental results on EMNIST comparing DFCA variants against centralized IFCA and decentralized D-FedAvg-M.

The algorithm also showed remarkable robustness to network sparsity. Testing revealed that a connection probability as low as 0.15 in an Erdős–Rényi graph is sufficient for convergence, with additional connectivity offering diminishing returns for model accuracy.

Discussion and Insights

DFCA's success hinges on its ability to maintain high accuracy without a central coordinator. The Local Initialization (DFCA-LI) strategy proved particularly robust in "noisy feature skew" scenarios. While it converges more slowly than its globally initialized counterpart, it often achieves a more stable final state when data distributions are messy or overlapping.

Limitations and Future Work

Despite its advantages, DFCA requires each client to store $k$ models. While manageable for small $k$ in typical IoT settings, this memory overhead could become restrictive for extremely resource-constrained devices if the number of clusters increases significantly. Additionally, the algorithm currently assumes the number of clusters is known a priori.

Future research will focus on quantifying wall-clock efficiency in physical hardware deployments and exploring methods for dynamic cluster discovery, allowing the network to adapt to an unknown or changing number of data distributions.

Conclusion

DFCA represents a significant step toward practical, decentralized machine learning for heterogeneous networks. By matching the performance of centralized coordinators through a robust, peer-to-peer gossip mechanism, it paves the way for more resilient and scalable IoT intelligence.

Summary updated

Abstract

Clustered Federated Learning has emerged as an effective approach for handling heterogeneous data across clients by partitioning them into clusters with similar or identical data distributions. However, most existing methods, including the Iterative Federated Clustering Algorithm (IFCA), rely on a central server to coordinate model updates which creates a bottleneck and single point of failure, limiting their applicability in more realistic decentralized learning settings. In this work, we introduce DFCA, a fully decentralized clustered Federated Learning algorithm that enables clients to collaboratively train cluster-specific models without central coordination. DFCA uses a sequential running average to aggregate models from neighbors as updates arrive, providing a communication-efficient alternative to batch aggregation while maintaining clustering performance. Our experiments on various datasets demonstrate that DFCA outperforms other decentralized algorithms and performs comparably to centralized IFCA, even under sparse connectivity, highlighting its robustness and practicality for dynamic real-world decentralized networks.
BibTeX
@misc{kirch2025dfca,
  author = {Jonas Kirch and Sebastian Becker and Tiago Koketsu Rodrigues and Stefan Harmeling},
  title = {{DFCA: Decentralized Federated Clustering Algorithm}},
  year = {2025},
  eprint = {2510.15300},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.15300},
}
Figure