Introduction
Standard Federated Learning (FL) architectures typically rely on a central server to coordinate model training across a network of clients. While effective, this centralized approach introduces a single point of failure and significant communication bottlenecks, particularly in Internet of Things (IoT) environments characterized by intermittent connectivity and limited bandwidth. Furthermore, real-world data is rarely Independent and Identically Distributed (IID). A single global model often fails to capture the nuances of heterogeneous data distributions across different device groups.
Clustered Federated Learning (CFL) addresses this by grouping similar clients to train specialized cluster models. However, existing state-of-the-art CFL methods, such as IFCA, still depend on a central coordinator. The Decentralized Federated Clustering Algorithm (DFCA) proposes a fully serverless, peer-to-peer alternative that maintains the benefits of clustering without the drawbacks of centralization.
Methodology: The DFCA Framework
DFCA enables clients to communicate exclusively with their immediate neighbors in a graph topology. The algorithm operates through a three-step iterative loop designed for decentralized environments.
1. Cluster Assignment
At the start of each round, a client evaluates all $k$ available cluster models against its local dataset. It performs local inference and self-assigns to the cluster model that yields the lowest empirical loss. This ensures that each device contributes to the model that best represents its specific data distribution.
2. Local Update
Once assigned, the client performs Stochastic Gradient Descent (SGD) to refine the chosen cluster model using its local data. Only the parameters of the assigned model are updated, preserving the specialization of the other $k-1$ models.
3. Decentralized Aggregation
The core innovation of DFCA lies in its "Sequential Aggregation" mechanism. Unlike traditional batch aggregation, which requires waiting for all neighbors to complete their updates, DFCA integrates neighbor updates as they arrive. By maintaining a sequential running average, the algorithm supports asynchronous communication and significantly reduces memory overhead. This makes it highly resilient to the "straggler" problem common in IoT networks.
The researchers evaluated two initialization strategies:
- DFCA-GI (Global Initialization): All clients begin with identical weights for all cluster models.
- DFCA-LI (Local Initialization): Clients start with personalized weights, which can enhance stability in highly noisy environments.
Experimental Evaluation
The performance of DFCA was benchmarked using several computer vision datasets modified to simulate real-world heterogeneity, including MNIST, EMNIST, FEMNIST, and CIFAR-10. For the MNIST and EMNIST datasets, digits were rotated (0°, 90°, 180°, 270°) to create synthetic clusters representing distinct data distributions.
Key Results
Experiments demonstrated that DFCA achieves performance parity with centralized state-of-the-art methods. On the EMNIST dataset with four clusters, DFCA-GI reached an accuracy of 85.7%, matching the centralized IFCA baseline. In contrast, other decentralized methods like FedSPD and D-FedAvg-M lagged behind, achieving significantly lower accuracy.
The algorithm also showed remarkable robustness to network sparsity. Testing revealed that a connection probability as low as 0.15 in an Erdős–Rényi graph is sufficient for convergence, with additional connectivity offering diminishing returns for model accuracy.
Discussion and Insights
DFCA's success hinges on its ability to maintain high accuracy without a central coordinator. The Local Initialization (DFCA-LI) strategy proved particularly robust in "noisy feature skew" scenarios. While it converges more slowly than its globally initialized counterpart, it often achieves a more stable final state when data distributions are messy or overlapping.
Limitations and Future Work
Despite its advantages, DFCA requires each client to store $k$ models. While manageable for small $k$ in typical IoT settings, this memory overhead could become restrictive for extremely resource-constrained devices if the number of clusters increases significantly. Additionally, the algorithm currently assumes the number of clusters is known a priori.
Future research will focus on quantifying wall-clock efficiency in physical hardware deployments and exploring methods for dynamic cluster discovery, allowing the network to adapt to an unknown or changing number of data distributions.
Conclusion
DFCA represents a significant step toward practical, decentralized machine learning for heterogeneous networks. By matching the performance of centralized coordinators through a robust, peer-to-peer gossip mechanism, it paves the way for more resilient and scalable IoT intelligence.
Summary updated