Edge Association Strategies for Synthetic Data Empowered Hierarchical Federated Learning With Non-IID Data

Jer Shyuan Ng, Aditya Pribadi Kalapaaking, Xiaoyu Xia, Dusit Niyato, Ibrahim Khalil, Iqbal Gondal

TL;DR

This paper introduces a synthetic-data-empowered Hierarchical Federated Learning (HFL) framework designed to tackle the twin challenges of data heterogeneity (non-IID data) and participant motivation. By using edge servers to distribute small amounts of synthetic data and employing evolutionary game theory to optimize how workers associate with those servers, the authors achieved significant accuracy gains—boosting model performance by several percentage points with as little as 5% additional synthetic data.

What Problem Does This Paper Address?

Standard Federated Learning (FL) allows devices to train models locally without sharing raw data, but it faces two major hurdles in the real world:

  1. Statistical Heterogeneity (Non-IID Data): Devices (workers) often have very different data distributions (e.g., one phone has only photos of cats, another only dogs). This "non-IID" nature slows down model convergence and hurts final accuracy.
  2. Communication and Participation: Workers may be far from the central server, leading to high latency or dropouts. Furthermore, training is resource-intensive; without proper incentives (rewards), workers may be reluctant to participate.

While Hierarchical FL (HFL) introduces an intermediate layer of edge servers to reduce latency, it doesn't inherently solve the data skew or the incentive problem. This paper aims to bridge that gap.

Methods

The authors propose a three-tier architecture: FL Workers $\rightarrow$ Edge Servers $\rightarrow$ Cloud Server. Their novel approach consists of two main components:

  • Synthetic Data Augmentation: Edge servers use pre-trained AI models (like cGANs or Latent Diffusion Models) to generate task-specific synthetic data. This data is distributed to workers, who mix it with their skewed local datasets. This "fills in the gaps" of the non-IID data, making the local training more representative of the global task.
  • Evolutionary Edge Association Game: To manage worker participation, the authors model the system as an evolutionary game.
  • The Utility Function: Workers choose which edge server to join based on a utility function that balances the rewards offered by the server against the costs of computation (training on local + synthetic data) and communication.
  • Replicator Dynamics: The paper uses differential equations to show how workers dynamically switch between edge servers to maximize their utility. They mathematically prove that this system reaches a Unique Evolutionary Equilibrium, meaning the network eventually settles into a stable state where no worker has an incentive to switch servers.

Datasets

The framework was evaluated using standard benchmarks and their synthetic counterparts:

  • MNIST: The classic 10-class handwritten digit dataset.
  • CIFAR-10: A 10-class object classification dataset.
  • CIFAKE: A synthetic version of CIFAR-10 generated using Latent Diffusion Models (LDMs).
  • cGAN-generated MNIST: Synthetic digit data created via Conditional Generative Adversarial Networks.

The authors tested these under extreme non-IID conditions, such as "Scenario 3," where each worker possesses data from only a single class.

Key Insights and Ideas

  • The "5% Rule": One of the most striking findings is that you don't need a lot of synthetic data to see a massive benefit. Adding just 5% synthetic data provided the most significant jump in accuracy. Beyond that, the authors observed diminishing returns, where adding 10% or 20% more data didn't significantly change the accuracy curves.
  • Self-Organizing Stability: By using evolutionary game theory, the authors show that the system can self-organize. Even if workers are "selfish" and only care about their own reward-to-cost ratio, the entire network naturally gravitates toward an optimal, stable distribution of workers across edge servers.
  • Mitigating the "Crowding Effect": The game model accounts for the fact that as more workers join a specific edge server, the individual share of the reward pool decreases. This naturally pushes workers toward less-congested servers, balancing the load across the hierarchy.

Key Results

  • Accuracy Boost: In the most extreme non-IID scenario (workers having only one class of data), the MNIST model accuracy jumped from 89.23% to 93.16% with just a 5% infusion of synthetic data.
  • CIFAR-10 Gains: On the more complex CIFAR-10 dataset, accuracy increased from roughly 0.70 to 0.84 when 25% synthetic data was used in a non-IID environment.
  • Convergence Speed: The simulation results confirmed that higher "learning rates" in the evolutionary game allowed workers to reach the stable equilibrium faster, though the final stable state remained the same regardless of the starting conditions.
  • Efficiency: Increasing the number of local updates ($\kappa_1$) at the worker level improved accuracy for a fixed number of cloud aggregations, suggesting that the framework allows for more efficient use of expensive long-distance communication.

Limitations and Future Work

  • Resource Constraints: While synthetic data helps accuracy, it increases the computational burden on workers (who now have more data to process). The authors acknowledge this cost in their utility function but note that it remains a trade-off.
  • Serverless Transition: The authors suggest that future work should explore a serverless approach to further reduce the reliance on a central FL server, potentially moving toward a fully decentralized edge intelligence model.
  • Pre-trained Model Dependency: The strategy relies on edge servers having access to high-quality pre-trained models to generate the synthetic data. In entirely new domains where no such models exist, this approach might face a "cold start" problem.

Summary updated

Abstract

In recent years, Federated Learning (FL) has emerged as a widely adopted privacy-preserving distributed training approach, attracting significant interest from both academia and industry. Research efforts have been dedicated to improving different aspects of FL, such as algorithm improvement, resource allocation, and client selection, to enable its deployment in distributed edge networks for practical applications. One of the reasons for the poor FL model performance is due to the worker dropout during training as the FL server may be located far away from the FL workers. To address this issue, an Hierarchical Federated Learning (HFL) framework has been introduced, incorporating an additional layer of edge servers to relay communication between the FL server and workers. While the HFL framework improves the communication between the FL server and workers, large number of communication rounds may still be required for model convergence, particularly when FL workers have non-independent and identically distributed (non-IID) data. Moreover, the FL workers are assumed to fully cooperate in the FL training process, which may not always be true in practical situations. To overcome these challenges, we propose a synthetic-data-empowered HFL framework that mitigates the statistical issues arising from non-IID local datasets while also incentivizing FL worker participation. In our proposed framework, the edge servers reward the FL workers in their clusters for facilitating the FL training process. To improve the performance of the FL model given the non-IID local datasets of the FL workers, the edge servers generate and distribute synthetic datasets to FL workers within their clusters. FL workers determine which edge server to associate with, considering the computational resources required to train on both their local datasets and the synthetic datasets.
BibTeX
@article{ng2025edge,
  author = {Jer Shyuan Ng and Aditya Pribadi Kalapaaking and Xiaoyu Xia and Dusit Niyato and Ibrahim Khalil and Iqbal Gondal},
  title = {{Edge Association Strategies for Synthetic Data Empowered Hierarchical Federated Learning With Non-IID Data}},
  journal = {IEEE Internet of Things Journal},
  year = {2025},
  doi = {10.1109/jiot.2025.3602382},
}

Loading related content...

Figure