Efficient Device Scheduling with Multi-Job Federated Learning

Chendi Zhou, Ji Liu, Juncheng Jia, Jingbo Zhou, Yang Zhou, Huaiyu Dai, Dejing Dou

Abstract

Recent years have witnessed a large amount of decentralized data in multiple (edge) devices of end-users, while the aggregation of the decentralized data remains difficult for machine learning jobs due to laws or regulations. Federated Learning (FL) emerges as an effective approach to handling decentralized data without sharing the sensitive raw data, while collaboratively training global machine learning models. The servers in FL need to select (and schedule) devices during the training process. However, the scheduling of devices for multiple jobs with FL remains a critical and open problem. In this paper, we propose a novel multi-job FL framework to enable the parallel training process of multiple jobs. The framework consists of a system model and two scheduling methods. In the system model, we propose a parallel training process of multiple jobs, and construct a cost model based on the training time and the data fairness of various devices during the training process of diverse jobs. We propose a reinforcement learning-based method and a Bayesian optimization-based method to schedule devices for multiple jobs while minimizing the cost. We conduct extensive experimentation with multiple jobs and datasets. The experimental results show that our proposed approaches significantly outperform baseline approaches in terms of training time (up to 8.67 times faster) and accuracy (up to 44.6% higher).

Introduction

Federated Learning (FL) allows models to be trained on decentralized data—such as photos on smartphones or sensor data on IoT devices—without that data ever leaving the device. While most FL research focuses on optimizing a single job, real-world servers often manage dozens of concurrent training tasks. This multi-job environment introduces two major challenges: the Straggler Problem, where heterogeneous device speeds bottleneck the process, and the Fairness-Accuracy Trade-off, where prioritizing fast devices can lead to biased models if those devices have non-representative (non-IID) data.

The MJ-FL Framework

The proposed Multi-Job Federated Learning (MJ-FL) framework introduces a parallel training process managed by a central server. At the heart of this system is a sophisticated cost model that combines Time Cost (latency of the slowest device in a round) and Data Fairness Cost (the variance in device selection frequency). By treating fairness as a technical requirement rather than just an ethical one, the framework ensures that global models converge accurately even in non-IID settings.

The MJ-FL system architecture, showcasing the interaction between the Scheduler, the multi-model Aggregator, and the distributed User pool.

To optimize this cost, the authors developed two distinct scheduling methods:

  • BODS (Bayesian Optimization-based Device Scheduling): Uses Gaussian Processes to predict optimal device combinations, proving highly efficient for simpler model architectures.
  • RLDS (Reinforcement Learning-based Device Scheduling): Employs a Long Short-Term Memory (LSTM) policy network to learn complex scheduling patterns for sophisticated models where dependencies are not immediately obvious.

Experimental Evaluation

The researchers evaluated MJ-FL using standard computer vision benchmarks (MNIST, Fashion-MNIST, EMNIST, and CIFAR-10) across 100 simulated devices. They tested various architectures, including VGG-16, ResNet-18, and LeNet-5, under both IID and non-IID data distributions.

Key Results

The MJ-FL approach significantly outperformed traditional baselines like Random selection, Greedy (fastest-first), and Genetic algorithms:

  • Significant Speedups: Training was up to 8.67 times faster for certain jobs. By considering the costs of all active jobs simultaneously, the scheduler keeps "idle" devices busy and minimizes total global training time.
  • Enhanced Accuracy: In non-IID scenarios, the proposed methods achieved up to 44.6% higher accuracy. This is because the fairness-aware cost model ensures a diverse set of data is included, preventing the model from "ignoring" the unique data held by slower devices.
Performance comparison showing test accuracy vs. elapsed time for VGG, CNN, and LeNet models under non-IID conditions.

Conclusion

MJ-FL demonstrates that multi-job efficiency in Federated Learning is best achieved by balancing speed with data representation. While BODS excels at simpler tasks, RLDS provides the necessary complexity to handle advanced neural networks. Future work may explore dynamic job prioritization and decentralized scheduling to further enhance the scalability of the framework.

Summary updated

Abstract

Recent years have witnessed a large amount of decentralized data in multiple (edge) devices of end-users, while the aggregation of the decentralized data remains difficult for machine learning jobs due to laws or regulations. Federated Learning (FL) emerges as an effective approach to handling decentralized data without sharing the sensitive raw data, while collaboratively training global machine learning models. The servers in FL need to select (and schedule) devices during the training process. However, the scheduling of devices for multiple jobs with FL remains a critical and open problem. In this paper, we propose a novel multi-job FL framework to enable the parallel training process of multiple jobs. The framework consists of a system model and two scheduling methods. In the system model, we propose a parallel training process of multiple jobs, and construct a cost model based on the training time and the data fairness of various devices during the training process of diverse jobs. We propose a reinforcement learning-based method and a Bayesian optimization-based method to schedule devices for multiple jobs while minimizing the cost. We conduct extensive experimentation with multiple jobs and datasets. The experimental results show that our proposed approaches significantly outperform baseline approaches in terms of training time (up to 8.67 times faster) and accuracy (up to 44.6% higher).
BibTeX
@inproceedings{zhou2022efficient,
  author = {Chendi Zhou and Ji Liu and Juncheng Jia and Jingbo Zhou and Yang Zhou and Huaiyu Dai and Dejing Dou},
  title = {{Efficient Device Scheduling with Multi-Job Federated Learning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  url = {https://openreview.net/forum?id=r8-m2azVS5D},
}
Figure