Resilience in Online Federated Learning: Mitigating Model-Poisoning Attacks via Partial Sharing

Ehsan Lari, Reza Arablouei, Vinay Chakravarthi Gogineni, Stefan Werner

TL;DR

This paper introduces PSO-Fed, an online federated learning algorithm that enhances resilience to model-poisoning attacks by sharing only a fraction of model updates. Surprisingly, this partial sharing, initially designed for communication efficiency, also significantly boosts robustness against malicious clients, leading to improved accuracy without extra computational cost.

What Problem Does This Paper Address?

Federated Learning (FL) allows multiple devices to collaboratively train a machine learning model without sharing their raw data, preserving privacy. However, FL is vulnerable to "model-poisoning attacks," where malicious clients deliberately corrupt their local model updates to degrade the global model's performance. Existing FL algorithms often struggle to balance communication efficiency with robustness against these attacks, especially in real-world scenarios with limited resources. This paper tackles the challenge of creating an FL algorithm that is both communication-efficient and highly resilient to model-poisoning attacks, particularly in an online learning setting where models are continuously updated.

Methods

The core of this work is the Partial-Sharing Online Federated Learning (PSO-Fed) algorithm. Unlike traditional FL where clients share their entire model updates, PSO-Fed employs a diagonal selection matrix to share only a subset of the model parameters with the server at each iteration. This partial sharing mechanism is the key innovation. The paper also introduces an intermittent model-poisoning attack model where Byzantine clients corrupt their updates with a certain probability.

The authors provide a theoretical analysis of PSO-Fed's convergence properties under these attacks, deriving formulas for its mean and mean-square error. A significant theoretical contribution is the identification of a non-trivial optimal stepsize that maximizes PSO-Fed's resistance to poisoning attacks, a departure from standard optimization where smaller steps are usually preferred.

Datasets

Not discussed in the paper. The experiments use synthetic data generated according to specified distributions and parameters.

Key Insights and Ideas

  • Partial Sharing as a Double-Edged Sword: The most striking insight is that partial sharing, primarily conceived to reduce communication overhead, unexpectedly acts as a powerful defense against model-poisoning attacks. This is a novel finding that offers a more elegant solution than adding separate defense mechanisms.
  • Optimal Stepsize for Robustness: The paper reveals that under model-poisoning attacks, there exists an optimal stepsize that minimizes the model's error. This is counter-intuitive to standard optimization where smaller steps are generally better for convergence and stability. This finding suggests a unique trade-off between convergence speed and attack resilience.
  • Theoretical Guarantees: The authors provide theoretical guarantees for PSO-Fed's convergence in both mean and mean-square senses, even in the presence of Byzantine clients. This rigorous analysis underpins the practical performance observed.
  • No Extra Client Burden: A crucial advantage highlighted is that PSO-Fed achieves enhanced robustness without imposing any additional computational or communication load on the participating clients, making it highly practical for resource-constrained devices.

Key Results

  • PSO-Fed consistently outperforms other leading FL algorithms (Online-Fed, SignSGD, CS-Fed, QS-Fed) in terms of steady-state Mean Squared Error (MSE) when facing model-poisoning attacks, as demonstrated in extensive simulations.
  • The performance of PSO-Fed degrades gracefully with an increasing number of Byzantine clients or higher attack strengths, but remains significantly better than Online-Fed.
  • Increasing the number of shared elements (M) generally leads to higher MSE, suggesting that less sharing (smaller M) enhances robustness.
  • The theoretical predictions for MSE and optimal stepsize closely match the simulation results, validating the analytical framework.
  • The paper identifies an approximate optimal stepsize (e.g., around 0.03) that yields the best performance under attack scenarios.

Limitations and Future Work

  • Assumptions: The theoretical analysis relies on several assumptions, including the stationarity of input data and independence of noise and attack signals. The authors acknowledge that deviations from these assumptions in real-world scenarios might affect theoretical predictions.
  • Linear Regression Task: The current work focuses on a linear regression task. Extending the analysis and methodology to more complex models (e.g., deep neural networks) and other domains (e.g., computer vision, natural language processing) is a promising future direction.
  • Other Adversarial Attacks: The paper primarily addresses model-poisoning attacks. Future work could investigate PSO-Fed's resilience against other types of adversarial attacks, such as data poisoning or label poisoning, and its performance under different network conditions.
  • Practical Deployment: While the algorithm is designed for efficiency, further research could explore its performance in highly dynamic or heterogeneous network environments with intermittent client availability.

Summary updated

Abstract

Federated learning (FL) allows training machine learning models on distributed data without compromising privacy. However, FL is vulnerable to model-poisoning attacks where malicious clients tamper with their local models to manipulate the global model. In this work, we investigate the resilience of the partial-sharing online FL (PSO-Fed) algorithm against such attacks. PSO-Fed reduces communication overhead by allowing clients to share only a fraction of their model updates with the server. We demonstrate that this partial sharing mechanism has the added advantage of enhancing PSO-Fed's robustness to model-poisoning attacks. Through theoretical analysis, we show that PSO-Fed maintains convergence even under Byzantine attacks, where malicious clients inject noise into their updates. Furthermore, we derive a formula for PSO-Fed's mean square error, considering factors like stepsize, attack probability, and the number of malicious clients. Interestingly, we find a non-trivial optimal stepsize that maximizes PSO-Fed's resistance to these attacks. Extensive numerical experiments confirm our theoretical findings and showcase PSO-Fed's superior performance against model-poisoning attacks compared to other leading FL algorithms.
BibTeX
@article{lari2025resilience,
  author = {Ehsan Lari and Reza Arablouei and Vinay Chakravarthi Gogineni and Stefan Werner},
  title = {{Resilience in Online Federated Learning: Mitigating Model-Poisoning Attacks via Partial Sharing}},
  journal = {IEEE Trans. Signal Inf. Process. Over Networks},
  year = {2025},
  doi = {10.1109/tsipn.2025.3559444},
}
Figure