Abstract
Federated Learning (FL) enables decentralized model training but remains vulnerable to Byzantine attacks, where malicious clients submit "poisoned" updates. This study explores Partial-Sharing Online Federated Learning (PSO-Fed) as a robust alternative. Originally designed for communication efficiency, partial sharing is shown to inherently dilute the impact of malicious updates. We provide a theoretical analysis of the Mean Square Error (MSE) under attack and demonstrate through simulations that PSO-Fed significantly outperforms existing robust aggregation techniques.
Introduction
The decentralized nature of Federated Learning creates a significant security vulnerability: Byzantine attacks. In these scenarios, malicious actors can steer the global model toward failure by injecting noise or biased updates. While robust aggregation methods like median-based filtering exist, they often impose high computational costs on resource-constrained IoT devices. This research, recently published in IEEE Transactions on Signal and Information Processing over Networks, investigates whether communication-efficient strategies, specifically partial sharing, can provide "out-of-the-box" resilience against such model-poisoning attacks.
Methodology: PSO-Fed
The proposed framework, Partial-Sharing Online Federated Learning (PSO-Fed), combines two key strategies:
- Partial Sharing: Instead of transmitting the full vector of $D$ model parameters, each client shares only a subset $M$ ($M < D$) per iteration.
- Online Learning: The model is updated continuously as new data streams arrive, ensuring the global model adapts to real-time changes.
The study models Byzantine clients as adversaries that inject zero-mean white Gaussian noise into their shared updates with a specific attack probability ($p_a$). By using block Kronecker products and vectorization, the authors derived a theoretical steady-state MSE, allowing for precise performance prediction based on network parameters and attack intensity.
Key Insights and Results
Sparsity as a Natural Defense
A primary finding is that partial sharing acts as a physical filter. By restricting the number of parameters shared by any single client, the "poison" introduced by a Byzantine actor is prevented from affecting the entire global model simultaneously. This "dilution" effect allows the network to maintain convergence even when a significant portion of the clients are compromised.
As shown in Figure 1, PSO-Fed maintains a significantly lower steady-state MSE compared to standard Online-Fed and other state-of-the-art (SOTA) methods like SignSGD, CS-Fed, and QS-Fed as the number of Byzantine clients increases. In a 100-client network, PSO-Fed can maintain a 5-10 dB advantage over its competitors.
The Non-Trivial Optimal Stepsize
In standard stochastic gradient descent (SGD), a smaller stepsize ($\mu$) typically leads to lower steady-state error. However, this research reveals a critical shift under adversarial conditions: there exists a non-zero optimal stepsize ($\mu^*$).
Figure 6 illustrates this phenomenon. If the stepsize is too small, the model becomes overly sensitive to the persistent noise from attackers; if it is too large, the system becomes unstable. Identifying this "sweet spot" is essential for maximizing the robustness of FL systems in hostile environments. Furthermore, the inclusion of a Stochastic Selection Algorithm (SSA) further enhances performance across various stepsize regimes.
Conclusion
PSO-Fed proves to be a dual-purpose solution, offering both communication efficiency and robust defense against model poisoning. The theoretical framework developed in this paper matches simulation results with high accuracy, providing a reliable tool for system designers. Future work will extend these findings to non-convex deep learning models and investigate resilience against more sophisticated, targeted backdoor attacks.
Summary updated