Abstract
In Federated Learning (FL), the limited accessibility of data from diverse locations and user types poses a significant challenge due to restricted user participation. Expanding client access and diversifying data enhance models by incorporating diverse perspectives, thereby enhancing adaptability. However, challenges arise in dynamic and mobile environments where certain devices may become inaccessible as FL clients, impacting data availability and client selection methods. To address this, we propose an On-Demand solution, deploying new clients using Docker Containers on-the-fly. Our On-Demand solution, employing Deep Reinforcement Learning (DRL), targets client availability and selection, while considering data shifts, and container deployment complexities. It employs an autonomous end-to-end solution for handling model deployment and client selection. The DRL strategy uses a Markov Decision Process (MDP) framework, with a Master Learner and a Joiner Learner. The designed cost functions represent the complexity of the dynamic client deployment and selection. Simulated tests show that our architecture can easily adjust to changes in the environment and respond to On-Demand requests. This underscores its ability to improve client availability, capability, accuracy, and learning efficiency, surpassing heuristic and tabular reinforcement learning solutions.
Introduction
Standard Federated Learning (FL) often assumes a stable pool of clients. However, in real-world dynamic environments—such as autonomous vehicle fleets or wearable health monitor networks—devices frequently move in and out of range. This instability leads to two primary issues: data scarcity and bias from missing mobile device perspectives, and a "software gap" where devices with relevant data lack the necessary libraries or APIs for training. Existing solutions using simple heuristics or random selection often fail to adapt to rapid data distribution shifts (non-IID data).
The On-Demand Framework
The proposed framework treats client participation as a deployment challenge. Instead of waiting for a device to be ready, the server pushes a lightweight Docker container to volunteer devices, encapsulating the model, training methods, and APIs.
System Architecture
The architecture is structured into three main layers: the Server, the Orchestrator, and the Volunteering Device. This hierarchy allows for efficient management of resources and intelligent client selection.
A dual-stage Deep Reinforcement Learning (DRL) strategy using Deep Q-Networks (DQN) manages this complexity. A Master Learner trains offline on historical data to establish a base model, while a Joiner Learner refines the strategy online to handle real-time environmental changes. The agent balances goals such as minimizing active hosts, maximizing data diversity, and adhering to physical hardware constraints like RAM and battery capacity.
Algorithmic Approach
The training process follows a structured round-based approach where clients are dynamically selected and deployed based on the DRL agent's decisions.
Experimental Evaluation
The framework was evaluated using the Mobile Data Challenge (MDC) dataset, which contains real-world movement records from 47 users. This dataset simulates realistic data shifts as users move between different environments, providing a rigorous test for the DRL agent's adaptability.
Key Results
- Rapid Convergence: The DRL approach reached target accuracy in under 10 rounds, significantly faster than tabular RL or heuristic methods.
- High Efficiency: The framework achieved a 92% acceptance rate for training rounds, compared to random selection which discarded over 60% of rounds due to poor client quality.
- Adaptability: The DRL agent successfully re-prioritized deployments during simulated data shifts, maintaining model performance where heuristic models saw sharp drops in accuracy.
Conclusion
By treating client mobility as an opportunity for proactive data scouting rather than a failure, the On-Demand FL framework significantly improves model adaptability and training efficiency. Future work will address security hardening for orchestrator nodes and explore multi-model deployment strategies to further optimize edge resource utilization.
Summary updated