Data-Heterogeneous Hierarchical Federated Learning with Mobility

Tan Chen, Jintao Yan, Yuxuan Sun, Sheng Zhou, Deniz Gündüz, Zhisheng Niu

TL;DR

While mobility is traditionally viewed as a hurdle in wireless communications, this paper demonstrates that it can actually be a "feature" for Hierarchical Federated Learning (HFL). By moving between different edge servers, vehicles naturally "mix" heterogeneous data, significantly improving model accuracy and convergence speed in non-i.i.d. (non-independent and identically distributed) scenarios.

What Problem Does This Paper Address?

Federated Learning (FL) allows AI models to be trained across many devices without sharing raw data, preserving privacy. However, in vehicular networks, two major challenges arise:

  1. Data Heterogeneity: Different vehicles collect different types of data based on their routes (e.g., one car sees mostly trucks, another sees mostly sedans). This "non-i.i.d." data causes local models to diverge, hurting the final global model's performance.
  2. The Mobility Paradox: In Hierarchical FL (where edge servers aggregate data from local vehicles before sending it to a central cloud), vehicle mobility is usually seen as a nuisance that causes unstable connections and changing network topologies.

The authors of this paper ask: Can we exploit vehicle mobility to actually improve learning?

Methods

The researchers propose an algorithm called Mob-HierFAVG, an extension of the Hierarchical Federated Averaging (HierFAVG) framework that explicitly accounts for mobility.

The core of their contribution is a rigorous convergence analysis. They derive a mathematical upper bound for the learning error that includes terms for both data heterogeneity and mobility. Key technical steps include:

  • Quantifying Heterogeneity: They use "gradient difference" to measure how much a local or edge-level model differs from the global objective.
  • Modeling Mobility as a "Mixer": They show that as vehicles move between the coverage areas of different edge servers, the "edge-level" data distribution becomes more uniform over time.
  • Virtual Parameters: To prove convergence, they introduce "virtual" cloud and centralized parameters to track the weighted sum of local models across the moving network.

Datasets

The authors evaluated their approach using:

  • CIFAR-10: A standard image recognition dataset. They partitioned the data into three scenarios: i.i.d. (uniform distribution), local non-i.i.d. (each vehicle has specific classes), and edge non-i.i.d. (each edge server's initial territory has specific classes).
  • SUMO (Simulation of Urban MObility): A professional-grade traffic simulator used to create a realistic Manhattan-style grid where vehicles move stochastically at various speeds ($v$).

Key Insights and Ideas

  • Mobility is a Natural Data Shuffler: In a static network, an edge server is stuck with whatever biased data its local devices have. In a mobile network, vehicles act as physical data carriers. As they move, they bring their unique data distributions to new edge servers, effectively "shuffling" the global dataset.
  • The "Edge Non-i.i.d." Advantage: The benefits of mobility are most pronounced when data is clustered geographically (edge non-i.i.d.). In these cases, mobility is the primary mechanism that prevents edge servers from training overly biased sub-models.
  • Diminishing Returns of Speed: While higher speeds lead to faster convergence, the benefit eventually saturates. Once vehicles move fast enough to sufficiently mix the data within a few aggregation rounds, further increases in speed provide negligible gains.

Key Results

  • Accuracy Boost: In the most challenging "edge non-i.i.d." scenario (where each edge server initially sees only one class of data), mobility improved final model accuracy by 15.1% (from 40.9% to 56.0%).
  • Faster Convergence: At a speed of 30 m/s, the model reached 75% accuracy 65.5% faster than a stationary network (142 epochs vs. 412 epochs).
  • Validation of Theory: The simulation results closely matched the derived convergence bound, confirming that reducing the "edge gradient difference" through mobility directly tightens the error bound.

Limitations and Future Work

  • Communication Overhead: The paper assumes a synchronized system and doesn't deeply explore the communication costs or packet loss associated with high-speed handovers between edge servers.
  • Simplified Mobility: While SUMO provides realistic traffic, the "Manhattan grid" is still a structured environment. Real-world performance might vary in rural or highly congested highway scenarios.
  • Future Directions: The authors suggest that future work could involve optimizing the frequency of edge and cloud aggregations dynamically based on the observed mobility rate of the vehicles to further save energy and bandwidth.

Summary updated

Abstract

Federated learning enables distributed training of machine learning (ML) models across multiple devices in a privacy-preserving manner. Hierarchical federated learning (HFL) is further proposed to meet the requirements of both latency and coverage. In this paper, we consider a data-heterogeneous HFL scenario with mobility, mainly targeting vehicular networks. We derive the convergence upper bound of HFL with respect to mobility and data heterogeneity, and analyze how mobility impacts the performance of HFL. While mobility is considered as a challenge from a communication point of view, our goal here is to exploit mobility to improve the learning performance by mitigating data heterogeneity. Simulation results verify the analysis and show that mobility can indeed improve the model accuracy by up to 15.1\% when training a convolutional neural network on the CIFAR-10 dataset using HFL.
BibTeX
@inproceedings{chen2023data,
  author = {Tan Chen and Jintao Yan and Yuxuan Sun and Sheng Zhou and Deniz Gündüz and Zhisheng Niu},
  title = {{Data-Heterogeneous Hierarchical Federated Learning with Mobility}},
  booktitle = {ITC},
  year = {2023},
  doi = {10.1109/itc-3560063.2023.10555630},
}

Loading related content...

Figure