Abstract
Data generated at the network edge can be processed locally by leveraging the paradigm of edge computing (EC). Aided by EC, decentralized federated learning (DFL), which overcomes the single-point-of-failure problem in the parameter server (PS) based federated learning, is becoming a practical and popular approach for machine learning over distributed data. However, DFL faces two critical challenges, i.e., system heterogeneity and statistical heterogeneity introduced by edge devices. To ensure fast convergence with the existence of slow edge devices, we present an efficient DFL method, termed FedHP, which integrates adaptive control of both local updating frequency and network topology to better support the heterogeneous participants. We establish a theoretical relationship between local updating frequency and network topology regarding model training performance and obtain a convergence upper bound. Upon this, we propose an optimization algorithm, that adaptively determines local updating frequencies and constructs the network topology, so as to speed up convergence and improve the model accuracy. Evaluation results show that the proposed FedHP can reduce the completion time by about 51% and improve model accuracy by at least 5% in heterogeneous scenarios, compared with the baselines.
Introduction
Decentralized Federated Learning (DFL) has emerged as a robust alternative to traditional Parameter Server (PS) architectures, eliminating single points of failure and communication bottlenecks by allowing edge devices to share models directly with peers. However, real-world deployment is hindered by system heterogeneity (varying hardware capabilities leading to "stragglers") and statistical heterogeneity (non-IID data distributions). Existing methods often address these issues in isolation, frequently sacrificing accuracy for training speed. We introduce FedHP (Federated Heterogeneous Participants), a framework designed to jointly optimize local update frequencies and network topology to mitigate these challenges.
The FedHP Approach
FedHP employs a dual-optimization strategy to balance computation and communication across heterogeneous nodes.
Adaptive Local Updating and Topology
Instead of a uniform update frequency, FedHP assigns a higher number of local training steps ($\tau$) to more powerful nodes, ensuring high resource utilization and reducing idle time. Simultaneously, the framework dynamically constructs the network topology by estimating "Consensus Distance"—a measure of model divergence between nodes. A greedy algorithm is then used to prune slow communication links while maintaining network connectivity and preventing model drift.
The system utilizes a lightweight coordinator to aggregate metadata (e.g., computing speeds and consensus distances) and calculate the optimal configuration for each training round, as shown in the algorithm above.
Experimental Evaluation
The performance of FedHP was evaluated using EMNIST, CIFAR-10, and IMAGE-100 datasets, modified to simulate class distribution skew. Results demonstrate that FedHP reduces total completion time by approximately 51% compared to standard decentralized SGD (D-PSGD) and specialized baselines like PENS or LD-SGD. Furthermore, in highly non-IID scenarios, FedHP improves test accuracy by at least 5%.
Discussion: The $\tau$ Trade-off
A key finding of this research is the existence of an optimal "Goldilocks" zone for local updates.
As illustrated in the figures above, too few local updates result in excessive communication overhead, while too many updates cause nodes to overfit to local data biases, degrading global accuracy. FedHP's adaptive mechanism successfully navigates this trade-off. Additionally, the results suggest that a well-tuned synchronous system can outperform asynchronous alternatives by avoiding the "stale model" issues prevalent in non-IID environments.
Conclusion and Future Work
FedHP effectively addresses the dual challenges of system and statistical heterogeneity in DFL. While the current implementation relies on a central coordinator for metadata, future research will focus on fully peer-to-peer decision-making and enhancing privacy protections for the shared consensus metadata.
Summary updated