Abstract
Federated Learning (FL) has promised a new approach to resolve the challenges in machine learning by bringing computation to the data. However, state-of-the-art systems often rely on a single-node aggregator that is insufficient for training large device populations or complex models. As model sizes and client counts increase, single-node aggregators face significant memory, computation, and communication bottlenecks. We propose a new aggregation service that classifies workloads and adaptively selects the best solution based on model size and client volume. Our system provides a fault-tolerant, robust, and efficient solution utilizing existing parallel and distributed frameworks. Through extensive evaluation, we demonstrate the shortcomings of static approaches and show how our elastic service outperforms current frameworks.
Introduction: The Aggregator Bottleneck
Federated Learning allows devices to train machine learning models locally, preserving privacy by keeping raw data on-device. The central challenge lies in the "aggregator"—the server responsible for fusing local updates into a global model. Most existing frameworks assume the aggregator resides in a cloud environment with virtually infinite resources.
However, deploying FL at the "Edge" (near IoT devices) introduces severe constraints. Edge data centers have limited CPU, memory, and bandwidth. Static aggregation methods—those that use a fixed resource allocation—fail in these environments; they are either too slow for large client pools or prohibitively expensive for small ones. This paper addresses the need for a resource-aware, elastic aggregation service that scales dynamically with the workload.
An Adaptive, Multi-Tiered Framework
The core innovation is a hybrid framework that utilizes a Q-learning-based Reinforcement Learning (RL) agent to select the optimal aggregation strategy for each training round. The agent monitors state variables such as the number of clients, model size, and available system resources to choose between three distinct methodologies:
- Numba-based (Multi-core): This tier is optimized for single-node performance. By using Numba to parallelize Python loops, the system achieves high efficiency for smaller workloads without the overhead of distributed systems.
- Spark-based (Multi-node): For large models and high client participation, the system leverages Apache Spark. This allows for horizontal scaling, distributing heavy I/O and decompression tasks across a cluster.
- Serverless (AWS Lambda): Utilizing a "tree-reduce" approach, this tier triggers massive numbers of short-lived functions. It is ideal for high-concurrency scenarios with many small updates, though it can become costly for very large models.
Experimental Evaluation
The system was evaluated using a variety of model architectures, including custom CNNs (from 4.6 MB to 956 MB) and standard benchmarks like ResNet50 and VGG16. A custom client emulator simulated up to 1 million clients across global AWS regions.
Performance Gains with Numba
For smaller client counts, the Numba-optimized approach significantly outperforms vanilla Python implementations. By parallelizing the fusion algorithm, the system reduces the CPU bottleneck often found at the aggregator.
Scaling to the Massive Edge
As the number of sampled clients grows into the tens of thousands, the Spark-based distributed approach becomes essential. The evaluation showed that while traditional methods might crash under edge resource limits, the proposed system maintains a linear scaling trend.
Key Insights
- CPU as the Primary Bottleneck: While network latency is a common concern in FL, the complexity of fusion algorithms like FedAvg creates a significant CPU bottleneck. Smart software scheduling and parallelization (like Numba) are often more cost-effective than specialized GPU hardware.
- The Decompression Tax: Model compression saves bandwidth but requires the aggregator to perform intensive decompression. The Spark-based method mitigates this by parallelizing decompression across worker nodes.
- No "One Size Fits All": The "best" method changes drastically based on the model. For small models, the adaptive method reduced latency by 88% simply by avoiding the overhead of Spark or Serverless functions.
Conclusion and Future Work
The proposed elastic aggregation service achieves up to an 8x improvement in time efficiency and a 2x reduction in operational costs compared to static approaches. By dynamically matching the aggregation strategy to the workload, the system enables scalable FL in resource-constrained edge environments.
Future research will focus on integrating non-synchronous fusion algorithms and exploring the use of specialized edge AI accelerators. Additionally, refining the RL agent to better handle "straggler" clients—devices with unpredictable reporting times—remains a priority for improving real-world robustness.
Summary updated