TL;DR
This paper introduces a novel defense mechanism called Robust Learning Rate (RLR) to protect Federated Learning (FL) models from backdoor attacks. RLR works by intelligently adjusting the server's learning rate based on the sign of agent updates, effectively neutralizing malicious attempts to embed backdoors while minimally impacting model accuracy.
What Problem Does This Paper Address?
Federated Learning (FL) enables collaborative model training across multiple devices without sharing raw data, making it attractive for privacy-sensitive applications. However, FL is vulnerable to backdoor attacks. In these attacks, an adversary manipulates the training process to embed a hidden functionality into the model. This backdoor can be triggered later by a specific pattern (the "trojan") to force the model into a desired misclassification, all while the model performs normally on clean data. This makes such attacks difficult to detect. Traditional defenses often struggle with these attacks in the decentralized FL setting, where only model updates, not raw data, are visible to the server.
Methods
The core of the proposed defense, Robust Learning Rate (RLR), lies in a simple yet effective modification of the aggregation server's learning rate. Instead of a uniform learning rate across all dimensions of the model's parameters, RLR dynamically adjusts it based on the sign of the aggregated updates for each dimension.
The intuition is that for a backdoor to be successfully embedded, the adversary's malicious updates must dominate the honest agents' updates in specific dimensions. RLR leverages this by introducing a "learning threshold" ($\theta$). If the sum of the signs of agent updates for a particular dimension falls below this threshold (indicating a lack of consensus or potential adversarial influence), RLR inverts the learning rate for that dimension. This effectively pushes the model away from the direction suggested by the potentially malicious updates, thereby maximizing the adversary's loss on that dimension and preventing the backdoor from taking hold.
The authors also provide a theoretical convergence rate analysis for their RLR scheme, demonstrating its stability and effectiveness.
Datasets
The paper evaluates its proposed defense on the following datasets:
- Fashion MNIST: Used for experiments in the i.i.d. (independent and identically distributed) setting.
- Federated EM-NIST: Used for experiments in the more realistic non-i.i.d. setting.
- CIFAR10: Used for testing against distributed backdoor attacks.
Key Insights and Ideas
- Sign-based defense is powerful: The paper demonstrates that focusing on the sign of model updates, rather than just their magnitude, can be a surprisingly effective way to detect and counteract adversarial influence in FL.
- Adversarial updates often lack consensus: The core conjecture is that adversarial updates will likely differ in direction from honest updates for at least some model parameters. RLR exploits this by inverting the learning rate when consensus is low.
- Minimal impact on accuracy: A key finding is that RLR effectively mitigates backdoor attacks with only a minor degradation in the model's overall accuracy on clean data, making it a practical defense.
- RLR is compatible with other defenses: The RLR mechanism can be combined with existing techniques like weight clipping and noise addition, further enhancing robustness.
Key Results
- Significant backdoor reduction: RLR consistently and significantly reduced backdoor accuracy across various settings (i.i.d., non-i.i.d., distributed attacks), often eliminating it entirely or reducing it to negligible levels.
- Minimal accuracy drop: The defense achieved this while maintaining high validation and base class accuracies, with drops typically less than 1-2% compared to non-attacked models.
- Outperforms existing methods: The paper shows that RLR outperforms several state-of-the-art defenses, particularly in scenarios where other methods fail.
- Effective against distributed attacks: RLR demonstrated robustness against a novel distributed backdoor attack where the trojan pattern is split across multiple adversarial agents.
Limitations and Future Work
- Hyperparameter tuning: The effectiveness of RLR relies on setting the learning threshold ($\theta$) appropriately. The paper notes that this parameter is non-trivial to tune and provides empirical guidance.
- Convergence speed: While RLR effectively prevents backdoors, it can sometimes slow down the convergence of the model. The authors suggest a strategy of activating RLR only towards the end of training or when a backdoor is suspected to mitigate this.
- Future work: The authors plan to analyze how RLR influences model performance in different non-i.i.d. settings, suggesting further exploration of its behavior in complex data distributions. They also aim to investigate how adversaries might attempt to bypass RLR.
Summary updated