Defending against Backdoors in Federated Learning with Robust Learning Rate

Mustafa Safa Özdayi, Murat Kantarcioglu, Yulia R. Gel

TL;DR

This paper introduces a novel defense mechanism called Robust Learning Rate (RLR) to protect Federated Learning (FL) models from backdoor attacks. RLR works by intelligently adjusting the server's learning rate based on the sign of agent updates, effectively neutralizing malicious attempts to embed backdoors while minimally impacting model accuracy.

What Problem Does This Paper Address?

Federated Learning (FL) enables collaborative model training across multiple devices without sharing raw data, making it attractive for privacy-sensitive applications. However, FL is vulnerable to backdoor attacks. In these attacks, an adversary manipulates the training process to embed a hidden functionality into the model. This backdoor can be triggered later by a specific pattern (the "trojan") to force the model into a desired misclassification, all while the model performs normally on clean data. This makes such attacks difficult to detect. Traditional defenses often struggle with these attacks in the decentralized FL setting, where only model updates, not raw data, are visible to the server.

Methods

The core of the proposed defense, Robust Learning Rate (RLR), lies in a simple yet effective modification of the aggregation server's learning rate. Instead of a uniform learning rate across all dimensions of the model's parameters, RLR dynamically adjusts it based on the sign of the aggregated updates for each dimension.

The intuition is that for a backdoor to be successfully embedded, the adversary's malicious updates must dominate the honest agents' updates in specific dimensions. RLR leverages this by introducing a "learning threshold" ($\theta$). If the sum of the signs of agent updates for a particular dimension falls below this threshold (indicating a lack of consensus or potential adversarial influence), RLR inverts the learning rate for that dimension. This effectively pushes the model away from the direction suggested by the potentially malicious updates, thereby maximizing the adversary's loss on that dimension and preventing the backdoor from taking hold.

The authors also provide a theoretical convergence rate analysis for their RLR scheme, demonstrating its stability and effectiveness.

Datasets

The paper evaluates its proposed defense on the following datasets:

  • Fashion MNIST: Used for experiments in the i.i.d. (independent and identically distributed) setting.
  • Federated EM-NIST: Used for experiments in the more realistic non-i.i.d. setting.
  • CIFAR10: Used for testing against distributed backdoor attacks.

Key Insights and Ideas

  • Sign-based defense is powerful: The paper demonstrates that focusing on the sign of model updates, rather than just their magnitude, can be a surprisingly effective way to detect and counteract adversarial influence in FL.
  • Adversarial updates often lack consensus: The core conjecture is that adversarial updates will likely differ in direction from honest updates for at least some model parameters. RLR exploits this by inverting the learning rate when consensus is low.
  • Minimal impact on accuracy: A key finding is that RLR effectively mitigates backdoor attacks with only a minor degradation in the model's overall accuracy on clean data, making it a practical defense.
  • RLR is compatible with other defenses: The RLR mechanism can be combined with existing techniques like weight clipping and noise addition, further enhancing robustness.

Key Results

  • Significant backdoor reduction: RLR consistently and significantly reduced backdoor accuracy across various settings (i.i.d., non-i.i.d., distributed attacks), often eliminating it entirely or reducing it to negligible levels.
  • Minimal accuracy drop: The defense achieved this while maintaining high validation and base class accuracies, with drops typically less than 1-2% compared to non-attacked models.
  • Outperforms existing methods: The paper shows that RLR outperforms several state-of-the-art defenses, particularly in scenarios where other methods fail.
  • Effective against distributed attacks: RLR demonstrated robustness against a novel distributed backdoor attack where the trojan pattern is split across multiple adversarial agents.

Limitations and Future Work

  • Hyperparameter tuning: The effectiveness of RLR relies on setting the learning threshold ($\theta$) appropriately. The paper notes that this parameter is non-trivial to tune and provides empirical guidance.
  • Convergence speed: While RLR effectively prevents backdoors, it can sometimes slow down the convergence of the model. The authors suggest a strategy of activating RLR only towards the end of training or when a backdoor is suspected to mitigate this.
  • Future work: The authors plan to analyze how RLR influences model performance in different non-i.i.d. settings, suggesting further exploration of its behavior in complex data distributions. They also aim to investigate how adversaries might attempt to bypass RLR.

Summary updated

Abstract

Federated learning (FL) allows a set of agents to collaboratively train a model without sharing their potentially sensitive data. This makes FL suitable for privacy-preserving applications. At the same time, FL is susceptible to adversarial attacks due to decentralized and unvetted data. One important line of attacks against FL is the backdoor attacks. In a backdoor attack, an adversary tries to embed a backdoor functionality to the model during training that can later be activated to cause a desired misclassification. To prevent backdoor attacks, we propose a lightweight defense that requires minimal change to the FL protocol. At a high level, our defense is based on carefully adjusting the aggregation server's learning rate, per dimension and per round, based on the sign information of agents' updates. We first conjecture the necessary steps to carry a successful backdoor attack in FL setting, and then, explicitly formulate the defense based on our conjecture. Through experiments, we provide empirical evidence that supports our conjecture, and we test our defense against backdoor attacks under different settings. We observe that either backdoor is completely eliminated, or its accuracy is significantly reduced. Overall, our experiments suggest that our defense significantly outperforms some of the recently proposed defenses in the literature. We achieve this by having minimal influence over the accuracy of the trained models. In addition, we also provide convergence rate analysis for our proposed scheme.
BibTeX
@inproceedings{ozdayi2021defending,
  author = {Mustafa Safa Özdayi and Murat Kantarcioglu and Yulia R. Gel},
  title = {{Defending against Backdoors in Federated Learning with Robust Learning Rate}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i10.17118},
}
Figure