Towards Cooperative Federated Learning over Heterogeneous Edge/Fog Networks

Su Wang, Seyyedali Hosseinalipour, Vaneet Aggarwal, Christopher G. Brinton, David J. Love, Weifeng Su, Mung Chiang

Introduction

Standard Federated Learning (FL) has traditionally relied on a rigid "star" topology, where a central server orchestrates training across a fleet of edge devices. While this model preserves data privacy, it often leaves edge resources underutilized and suffers from significant bottlenecks. Resource-constrained devices become "stragglers," slowing down the entire training round, while powerful local infrastructure like edge servers and routers often sit idle. Furthermore, connectivity gaps can exclude devices without a direct link to the central server from the learning process entirely.

To address these challenges, this paper introduces Cooperative Federated Learning (CFL). CFL breaks the star topology by leveraging device-to-device (D2D) and device-to-server (D2S) interactions. By treating the network as a pool of shared resources, CFL mitigates the straggler effect and improves both energy and temporal efficiency in heterogeneous edge/fog networks.

The CFL Framework: Two Pillars of Cooperation

The CFL paradigm is built upon two primary cooperative mechanisms designed to balance computational load and optimize communication.

1. Device-to-Device (D2D) Cooperation

D2D cooperation allows neighboring devices to assist each other in the training process. This is achieved through two main techniques:

  • Dataset Stratification: To prevent bias when sharing data between devices, CFL employs stratification. Local data is clustered into representative "strata," and only the most significant samples are shared. This ensures that the receiving device maintains a statistically balanced view of the data.
  • Local Model Offloading: Devices can segment their local models and transmit parts to nearby, better-connected neighbors. These neighbors act as relays, aggregating updates before they reach the central server.
Figure 1: Conceptual overview of Device-to-Device (D2D) cooperation mechanisms, including stratification and model segment sharing.

2. Device-to-Server (D2S) Cooperation

CFL also optimizes the interaction between devices and local edge infrastructure:

  • Load Balancing: Resource-constrained devices can offload raw data processing or intermediate training tasks to nearby edge servers, such as road-side units in smart city environments.
  • Floating Aggregation Points: Instead of a fixed central server, CFL dynamically selects aggregation points based on device mobility and network congestion, minimizing communication latency.
  • Efficient Routing: Network routers and switches are utilized to manage the flow of model parameters, adapting to real-time channel conditions.
Figure 4: Device-to-Server (D2S) cooperation illustrating dynamic aggregation server selection and data/model sharing.

Experimental Evaluation

The authors evaluated the CFL framework against state-of-the-art FL algorithms (e.g., FedAvg and Nova) using three standard benchmarks: MNIST, Fashion-MNIST, and CIFAR-10.

Key Results

  • Energy Efficiency: CFL demonstrated consistent energy savings, particularly on complex datasets like Fashion-MNIST and CIFAR-10, by optimizing task distribution.
  • Training Speed: The framework achieved target classification accuracy significantly faster than baseline methods. By reducing "temporal sensitivity," CFL ensures that models converge within strict time constraints.
  • System Robustness: D2D offloading successfully integrated data from devices that would otherwise be disconnected or categorized as stragglers due to poor direct links to the central server.

Discussion and Future Directions

The core insight of CFL is the transition from isolated silos to a collaborative resource pool. However, several challenges remain for future research:

  • Mobility: Maintaining cooperation in non-stationary networks where devices (e.g., drones or vehicles) frequently move in and out of range.
  • Unlabeled Data: Integrating self-supervised learning to utilize the vast amounts of unlabeled "dark data" at the edge.
  • Privacy Heterogeneity: Developing "cliques of trust" to handle varying privacy requirements among different users and devices.

By breaking the star topology, Cooperative Federated Learning paves the way for more resilient and efficient decentralized machine learning in the next generation of edge networks.

Summary updated

BibTeX
@article{wang2023towards,
  author = {Su Wang and Seyyedali Hosseinalipour and Vaneet Aggarwal and Christopher G. Brinton and David J. Love and Weifeng Su and Mung Chiang},
  title = {{Towards Cooperative Federated Learning over Heterogeneous Edge/Fog Networks}},
  journal = {IEEE Communications Magazine},
  year = {2023},
  doi = {10.1109/mcom.005.2200925},
}
Figure