Authors: Yichen Ruan, Xiaoxi Zhang, Shu-Che Liang, and Carlee Joe-Wong
Abstract
Standard Federated Learning (FL) assumes reliable device participation, yet real-world mobile hardware is often "flaky"—dropping out, sending partial updates, or joining late. This paper addresses these challenges by introducing an adaptive aggregation scheme (Scheme C) that reweights updates based on completed work and a "fast-reboot" strategy for late-arriving devices. We provide a rigorous convergence analysis and demonstrate through experiments on non-IID data that our approach significantly improves model accuracy and stability compared to traditional methods that discard or misweight partial updates.
Introduction: Beyond the "Plugged-In" Pipe Dream
Federated Learning (FL) allows a central coordinator to train a model across thousands of user devices without accessing raw data. However, algorithms like FedAvg rely on a fragile assumption: that every selected device stays connected and completes all local training rounds. In reality, mobile devices are constrained by battery life and connectivity. When a device fails to complete its task, traditional FL either discards the work or treats it as a full update, introducing significant bias, especially in non-IID (non-independently and identically distributed) settings.
The Challenge of Flaky Devices
"Flakiness" manifests as incompleteness (partial work), inactivity (temporary silence), early departures, and late arrivals. If a coordinator only aggregates updates from devices that finish all work, the model becomes biased toward users with powerful hardware or stable connectivity. If these users have specific data patterns, the model fails to generalize.
Proposed Schemes for Flexible Participation
The authors propose three key interventions to handle device heterogeneity:
- Scheme C (Adaptive Aggregation): Unlike standard methods that weight updates by data size, Scheme C scales updates based on the work actually finished. For instance, if a device completes 3 out of 5 local epochs, its update is scaled by 5/3. This "debiasing" ensures slow devices contribute their fair share to the global gradient.
- Fast-Reboot for Arrivals: When a new device joins late, the authors propose a "fast-reboot" where the coordinator applies extra gradient updates from the new device to quickly integrate its data distribution into the mature global model.
- Objective Shift Criteria: A mathematical threshold helps operators decide whether to maintain performance for departed users or shift the objective to focus on remaining ones.
Experimental Evaluation
The methods were evaluated on MNIST, EMNIST, and synthetic datasets modified for non-IID conditions. To ensure realism, the researchers used hardware traces from Raspberry Pis running FL tasks alongside CPU-intensive "competitor" processes.
Key Results and Insights
- Accuracy Gains: In highly heterogeneous settings, Scheme C significantly outperformed others. For EMNIST, it improved test accuracy by up to 4.8% and showed much higher stability than schemes that discarded incomplete work.
- Faster Recovery: The Fast-Reboot strategy allowed models to recover accuracy much faster after a new device joined, often in just 1 epoch compared to 4+ for vanilla methods.
- Convergence Guarantee: The authors proved that Scheme C is guaranteed to converge to the global optimum as long as devices are not inactive indefinitely.
Conclusion
This work makes FL more practical by accounting for the inherent unreliability of edge devices. While challenges remain in estimating participation history for new users and handling massive waves of arrivals, the proposed adaptive weighting and reboot strategies provide a robust foundation for real-world federated systems.
Summary updated