Prophet: Proactive Candidate-Selection for Federated Learning by Predicting the Qualities of Training and Reporting Phases

Huawei Huang, Kangying Lin, Song Guo, Pan Zhou, Zibin ZhengarXiv 2020 · OpenReview

TL;DR

This paper introduces "Prophet," a novel proactive candidate selection strategy for Federated Learning (FL). By predicting the future quality of mobile devices' training and reporting capabilities using LSTM and optimizing this selection with Deep Reinforcement Learning (DRL), Prophet significantly improves the efficiency and accuracy of FL, especially in dynamic mobile environments.

What Problem Does This Paper Address?

Federated Learning (FL) allows multiple devices to collaboratively train a global model without sharing their private data. While 5G/B5G networks alleviate communication latency, training latency remains a bottleneck. A primary cause of this latency is the "candidate selection" phase, where the FL server chooses which devices will participate in training. Existing methods are "reactive," meaning they select participants based solely on their current, observed resources. In dynamic mobile environments, these resources can change drastically, leading to selected devices failing to complete training or reporting their results. This results in wasted effort and reduced overall FL performance. The paper addresses this by proposing a "proactive" approach that predicts future device capabilities to make more reliable selection decisions.

Methods

The core of Prophet lies in a two-pronged approach:

  1. Proactive Prediction: Each mobile device (MU) locally uses a Long Short-Term Memory (LSTM) network to predict its future network connection quality and available CPU capacity for the upcoming training and reporting phases. This prediction is crucial because it moves beyond the "snapshot" of current resources.
  2. Deep Reinforcement Learning (DRL) for Selection: A Deep Reinforcement Learning (DRL) agent, specifically a Double DQN (DDQN) framework, is employed by the FL parameter server. This agent takes the predicted future states (from the LSTMs) and current states as input and learns an optimal policy for selecting up to K participants. The DRL agent is trained to maximize the number of "valid participants" (those who successfully complete training and reporting) and minimize penalties for failed participants.

Datasets

  • Custom Trace Dataset: The authors developed an Android app called "Tracer" to collect real-world trace data from approximately 100 student volunteers over six weeks on a university campus. This dataset includes GPS coordinates (for mobility patterns), cellular signal parameters (RSSI, RSRP, RSRQ, SINR), and CPU workload consumed by top apps. This custom dataset is a key contribution, as it captures the dynamic nature of mobile user behavior.
  • Standard Datasets: For evaluating the aggregated global FL model's test accuracy, the paper uses two standard image datasets:
  • MNIST
    • CIFAR-10 These datasets were used in a non-independently and identically distributed (non-i.i.d.) manner, simulating real-world scenarios where each device has a partial and biased view of the data.

Key Insights and Ideas

  • Predicting the Future is Key: The fundamental insight is that reactive selection based on current resources is inherently flawed in dynamic environments. Proactively predicting future device states is essential for robust FL.
  • LSTM for Mobility and Resource Prediction: LSTM networks are well-suited for capturing sequential patterns in user mobility and app usage, which directly influence future resource availability.
  • DRL for Adaptive Decision-Making: DRL provides a powerful framework for learning optimal selection policies in complex, dynamic environments where explicit rule-based systems are insufficient. The agent learns to balance exploiting known good participants with exploring new ones that might offer better future rewards.
  • Custom Data is Crucial: The use of a real-world, custom-collected trace dataset is vital for validating the approach's effectiveness in realistic, highly uncertain mobile scenarios.
  • Proactive Selection Benefits from "Bad" Current States: The paper demonstrates that Prophet can identify devices that might have poor current resources but are predicted to improve, leading to a more balanced and effective participant pool.

Key Results

  • Numerical Reward: Prophet (DDQN-based) achieved a numerical reward close to the optimal offline solution, significantly outperforming reactive baselines like FedCS and the simple FedAvg.
  • Ratio of Valid Participants: Prophet consistently achieved the highest ratio of valid participants (up to 95%), meaning more selected devices successfully completed their FL tasks.
  • Test Accuracy: In experiments with non-i.i.d. MNIST and CIFAR-10 datasets, Prophet achieved 10%-20% higher test accuracy compared to reactive baselines like FedCS, demonstrating the improved quality of the aggregated global model.
  • Adaptability to Uncertainties: When simulated unexpected events (like cellular network crashes) occurred, Prophet's reward temporarily dropped but quickly recovered, showcasing its ability to adapt its policy to changing conditions.

Limitations and Future Work

  • LSTM Prediction Accuracy: While Prophet's overall performance is superior, the paper acknowledges that LSTM prediction accuracies for POI and Sojourn duration are not extremely high (around 90% and 82% respectively). However, they argue that even with these accuracies, the proactive approach yields better results.
  • Computational Overhead: The local LSTM prediction and the DRL agent's decision-making introduce computational overhead on both the mobile devices and the server.
  • Future Work: The paper suggests that future research could explore more sophisticated prediction models and DRL algorithms. Further investigation into the impact of different types of uncertainties and the scalability of the approach to even larger numbers of devices would also be valuable.

Summary updated

Abstract

Although the challenge of the device connection is much relieved in 5G networks, the training latency is still an obstacle preventing Federated Learning (FL) from being largely adopted. One of the most fundamental problems that lead to large latency is the bad candidate-selection for FL. In the dynamic environment, the mobile devices selected by the existing reactive candidate-selection algorithms very possibly fail to complete the training and reporting phases of FL, because the FL parameter server only knows the currently-observed resources of all candidates. To this end, we study the proactive candidate-selection for FL in this paper. We first let each candidate device predict the qualities of both its training and reporting phases locally using LSTM. Then, the proposed candidateselection algorithm is implemented by the Deep Reinforcement Learning (DRL) framework. Finally, the real-world trace-driven experiments prove that the proposed approach outperforms the existing reactive algorithms
BibTeX
@misc{huang2020prophet,
  author = {Huawei Huang and Kangying Lin and Song Guo and Pan Zhou and Zibin Zheng},
  title = {{Prophet: Proactive Candidate-Selection for Federated Learning by Predicting the Qualities of Training and Reporting Phases}},
  year = {2020},
  eprint = {2002.00577},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2002.00577},
}
Figure