Abstract
In recent years, Federated Learning (FL) has emerged as a widely adopted privacy-preserving distributed training approach, attracting significant interest from both academia and industry. Research efforts have been dedicated to improving different aspects of FL, such as algorithm improvement, resource allocation, and client selection, to enable its deployment in distributed edge networks for practical applications. One of the reasons for the poor FL model performance is due to the worker dropout during training as the FL server may be located far away from the FL workers. To address this issue, an Hierarchical Federated Learning (HFL) framework has been introduced, incorporating an additional layer of edge servers to relay communication between the FL server and workers. While the HFL framework improves the communication between the FL server and workers, large number of communication rounds may still be required for model convergence, particularly when FL workers have non-independent and identically distributed (non-IID) data. Moreover, the FL workers are assumed to fully cooperate in the FL training process, which may not always be true in practical situations. To overcome these challenges, we propose a synthetic-data-empowered HFL framework that mitigates the statistical issues arising from non-IID local datasets while also incentivizing FL worker participation. In our proposed framework, the edge servers reward the FL workers in their clusters for facilitating the FL training process. To improve the performance of the FL model given the non-IID local datasets of the FL workers, the edge servers generate and distribute synthetic datasets to FL workers within their clusters. FL workers determine which edge server to associate with, considering the computational resources required to train on both their local datasets and the synthetic datasets.
Introduction
Standard Federated Learning (FL) allows distributed devices to train models locally without sharing raw data, preserving privacy while leveraging decentralized compute. However, deploying FL in real-world edge networks introduces two significant hurdles: statistical heterogeneity (non-IID data) and participation incentives. When workers possess highly skewed data distributions, model convergence slows and global accuracy suffers. Furthermore, because training is resource-intensive, workers may be reluctant to participate without adequate rewards.
While Hierarchical Federated Learning (HFL) introduces an intermediate layer of edge servers to reduce communication latency and mitigate worker dropout, it does not inherently solve the data skew or the incentive problem. This paper proposes a synthetic-data-empowered HFL framework designed to bridge these gaps by augmenting local datasets and optimizing worker-to-server association through evolutionary game theory.
System Architecture
The proposed framework utilizes a three-tier architecture: FL Workers → Edge Servers → Cloud Server. This hierarchy allows for localized model aggregation at the edge before global synchronization at the cloud, reducing the communication burden on the central server.
Synthetic Data Augmentation
To combat the performance degradation caused by non-IID data, edge servers utilize pre-trained generative models—such as Conditional Generative Adversarial Networks (cGANs) or Latent Diffusion Models (LDMs)—to produce task-specific synthetic data. This data is distributed to workers, who integrate it with their skewed local datasets. This augmentation "fills the gaps" in the local data distribution, ensuring that local model updates are more representative of the global task.
Evolutionary Edge Association Game
Managing worker participation requires a robust incentive and association strategy. The system is modeled as an evolutionary game where workers are "selfish" agents seeking to maximize their utility:
- The Utility Function: Workers evaluate edge servers based on the rewards offered for participation versus the costs of computation (training on both local and synthetic data) and communication.
- Replicator Dynamics: Using differential equations, the framework models how workers dynamically switch between edge servers to optimize their utility. As more workers join a specific server, the individual share of the reward pool decreases (the "crowding effect"), naturally pushing workers toward less-congested servers.
- Evolutionary Equilibrium: The paper mathematically proves that this system reaches a Unique Evolutionary Equilibrium. In this stable state, the network reaches an optimal distribution of workers across edge servers, and no worker has an incentive to change their association.
Experimental Evaluation
The framework was evaluated using standard benchmarks and their synthetic counterparts, including MNIST and CIFAR-10, alongside synthetic versions like CIFAKE (generated via LDMs) and cGAN-generated MNIST. Tests focused on extreme non-IID conditions, such as scenarios where each worker possesses data from only a single class.
Key Results
- The "5% Rule": Experiments revealed that massive amounts of synthetic data are not required for significant gains. Adding just 5% synthetic data provided the most substantial jump in accuracy. Beyond this threshold, the system showed diminishing returns, with 10% or 20% infusions yielding marginal additional benefits.
- Accuracy Boosts: In extreme non-IID scenarios, MNIST model accuracy increased from 89.23% to 93.16% with a 5% synthetic data infusion. On the more complex CIFAR-10 dataset, accuracy rose from approximately 0.70 to 0.84 when 25% synthetic data was utilized.
- Convergence and Efficiency: Simulation results confirmed that the evolutionary game reaches stability regardless of initial conditions. Furthermore, increasing the number of local updates at the worker level improved accuracy for a fixed number of cloud aggregations, suggesting the framework optimizes expensive long-distance communication.
Discussion and Future Work
While the infusion of synthetic data significantly improves accuracy, it does increase the computational burden on edge workers. The proposed utility function accounts for this trade-off, but it remains a primary constraint for resource-limited devices. Additionally, the strategy assumes edge servers have access to high-quality pre-trained models; domains lacking such models may face a "cold start" problem.
Future research will explore transitioning toward a serverless approach to further decentralize the hierarchy and reduce reliance on a central cloud server, moving closer to a fully autonomous edge intelligence model.
Summary updated