Data-Heterogeneous Hierarchical Federated Learning with Mobility

Tan Chen, Jintao Yan, Yuxuan Sun, Sheng Zhou, Deniz Gündüz, Zhisheng Niu

Abstract

Federated learning enables distributed training of machine learning (ML) models across multiple devices in a privacy-preserving manner. Hierarchical federated learning (HFL) is further proposed to meet the requirements of both latency and coverage. In this paper, we consider a data-heterogeneous HFL scenario with mobility, mainly targeting vehicular networks. We derive the convergence upper bound of HFL with respect to mobility and data heterogeneity, and analyze how mobility impacts the performance of HFL. While mobility is considered as a challenge from a communication point of view, our goal here is to exploit mobility to improve the learning performance by mitigating data heterogeneity. Simulation results verify the analysis and show that mobility can indeed improve the model accuracy by up to 15.1% when training a convolutional neural network on the CIFAR-10 dataset using HFL.

Introduction

Federated Learning (FL) has emerged as a powerful paradigm for training AI models across distributed devices without compromising data privacy. In vehicular networks, Hierarchical Federated Learning (HFL) is often employed, where edge servers aggregate data from local vehicles before communicating with a central cloud. However, two major hurdles persist: data heterogeneity (non-i.i.d. data) and the "mobility paradox."

Traditionally, vehicle mobility is viewed as a nuisance that causes unstable connections and changing network topologies. This paper challenges that view, demonstrating that mobility can be a "feature" rather than a bug. By moving between different edge servers, vehicles naturally "mix" heterogeneous data, significantly improving model accuracy and convergence speed in non-i.i.d. scenarios.

System Model and Methods

The researchers propose a three-tier architecture consisting of vehicles, edge servers (e.g., base stations), and a central cloud. This structure is designed to balance the low latency of edge computing with the global coverage of cloud computing.

Hierarchical Federated Learning architecture in a vehicular network.

The core contribution is the Mob-HierFAVG algorithm, an extension of the Hierarchical Federated Averaging framework that explicitly accounts for mobility. The authors provide a rigorous convergence analysis, deriving a mathematical upper bound for learning error that includes terms for both data heterogeneity and mobility. Key technical insights include:

  • Quantifying Heterogeneity: Using "gradient difference" to measure divergence between local, edge, and global objectives.
  • Mobility as a Mixer: Showing that as vehicles move, the "edge-level" data distribution becomes more uniform over time, effectively shuffling the global dataset.

To evaluate the approach, the authors used the CIFAR-10 image recognition dataset partitioned into various non-i.i.d. scenarios. Realistic traffic patterns were generated using the SUMO (Simulation of Urban MObility) simulator, creating a Manhattan-style grid where vehicles move at various speeds ($v$).

Experimental Results

The benefits of mobility are most pronounced when data is clustered geographically (edge non-i.i.d.). In these cases, mobility serves as the primary mechanism preventing edge servers from training overly biased sub-models.

Impact of mobility on test accuracy across different data heterogeneity scenarios.

Key Findings:

  • Accuracy Boost: In the most challenging "edge non-i.i.d." scenario—where each edge server initially sees only one class of data—mobility improved final model accuracy by 15.1% (from 40.9% to 56.0%).
  • Faster Convergence: Mobility significantly accelerates the training process. At a speed of 30 m/s, the model reached 75% accuracy 65.5% faster than a stationary network (142 epochs vs. 412 epochs).
  • Diminishing Returns of Speed: While higher speeds lead to faster convergence, the benefit eventually saturates once vehicles move fast enough to sufficiently mix data within a few aggregation rounds.

Conclusion and Future Work

This work demonstrates that vehicle mobility can be exploited to mitigate the negative effects of data heterogeneity in HFL. By acting as physical data carriers, moving vehicles naturally balance the data distribution across edge servers.

Future research directions include addressing the communication overhead and packet loss associated with high-speed handovers. Additionally, the authors suggest optimizing the frequency of edge and cloud aggregations dynamically based on observed mobility rates to further enhance efficiency in real-world vehicular environments.

Summary updated

Abstract

Federated learning enables distributed training of machine learning (ML) models across multiple devices in a privacy-preserving manner. Hierarchical federated learning (HFL) is further proposed to meet the requirements of both latency and coverage. In this paper, we consider a data-heterogeneous HFL scenario with mobility, mainly targeting vehicular networks. We derive the convergence upper bound of HFL with respect to mobility and data heterogeneity, and analyze how mobility impacts the performance of HFL. While mobility is considered as a challenge from a communication point of view, our goal here is to exploit mobility to improve the learning performance by mitigating data heterogeneity. Simulation results verify the analysis and show that mobility can indeed improve the model accuracy by up to 15.1\% when training a convolutional neural network on the CIFAR-10 dataset using HFL.
BibTeX
@inproceedings{chen2023data,
  author = {Tan Chen and Jintao Yan and Yuxuan Sun and Sheng Zhou and Deniz Gündüz and Zhisheng Niu},
  title = {{Data-Heterogeneous Hierarchical Federated Learning with Mobility}},
  booktitle = {ITC},
  year = {2023},
  doi = {10.1109/itc-3560063.2023.10555630},
}
Figure