Invariant Aggregator for Defending against Federated Backdoor Attacks

Xiaoyang Wang, Dimitrios Dimitriadis, Oluwasanmi O Koyejo, Shruti Tople

TL;DR

This paper introduces an "Invariant Aggregator" for federated learning that effectively defends against backdoor attacks. By focusing on gradient directions that are consistently beneficial across clients (invariant directions) and filtering out those that might be manipulated by malicious actors, the approach significantly reduces attack success rates with minimal impact on model accuracy.

What Problem Does This Paper Address?

Federated learning (FL) allows multiple parties to collaboratively train a machine learning model without sharing their raw data. This is great for privacy, but it opens the door to adversarial attacks. One particularly insidious threat is "backdoor attacks," where malicious clients inject poisoned data with a specific "trigger." When this trigger is present in new, unseen data, the model makes an incorrect prediction, as dictated by the attacker. While defenses exist for attacks that degrade overall model performance, those that only affect predictions on triggered samples (backdoor attacks) are harder to detect and mitigate, especially in FL. This paper tackles the challenge of defending FL models against these sophisticated backdoor attacks, particularly in scenarios where the model's loss landscape is "flat," making it easier for attackers to subtly influence the training process.

Methods

The core of the proposed defense is the "Invariant Aggregator," which combines two key techniques:

  • AND-Mask: This component analyzes the "sign consistency" of each dimension across client updates (pseudo-gradients). If the signs of a gradient dimension are not consistent across a majority of clients, it suggests that this direction might be manipulated by a few malicious clients. The AND-mask then zeros out these inconsistent dimensions, effectively ignoring them in the aggregation. This focuses the aggregation on directions that are "invariant" – consistently beneficial across most clients.
  • Trimmed-Mean Estimator: To further enhance robustness, the AND-mask is complemented by a trimmed-mean estimator. This technique removes a certain percentage of the largest and smallest gradient values (outliers) before computing the mean. This helps to mitigate the influence of any remaining malicious updates that might have slipped through the AND-mask.

The authors argue that this combination is crucial: the AND-mask handles the directionality and avoids being fooled by flat loss landscapes, while the trimmed-mean deals with potential outliers.

Datasets

The paper evaluates its approach on three diverse datasets:

  • CIFAR-10: A widely used dataset for image classification.
  • Twitter: A dataset for sentiment analysis.
  • Phishing Emails: A tabular dataset for spam detection.

These datasets represent different modalities (visual, text, tabular) and are used to demonstrate the generality of the proposed defense.

Key Insights and Ideas

  • Flat Loss Landscapes are a Double-Edged Sword: While flat loss landscapes are generally good for optimization, they can inadvertently aid backdoor attacks. They allow malicious clients to subtly alter model parameters without significantly impacting the loss on benign data, making their malicious updates harder to distinguish from benign ones.
  • Gradient Direction Over Magnitude: The proposed defense prioritizes the direction of gradient updates (sign consistency) over their magnitude. This is key to overcoming the challenge posed by flat loss landscapes, where gradient magnitudes can be small and misleading.
  • Necessity of Combination: The paper's ablation studies show that neither the AND-mask nor the trimmed-mean estimator alone is sufficient. Their combined effect is what provides robust defense against backdoor attacks.
  • Focus on Invariance: The core idea is to aggregate updates that are "invariant" – consistently beneficial across a large number of clients – rather than those that might be specific to a few, potentially malicious, clients.

Key Results

  • Significant ASR Reduction: The proposed Invariant Aggregator reduces the backdoor attack success rate (ASR) by an average of 61.6% across experiments.
  • Negligible Utility Loss: The defense achieves this mitigation with a minimal drop in main task accuracy (Acc), averaging only a 1.2% loss compared to standard federated averaging (FedAvg).
  • Robustness Across Attacks and Datasets: The method demonstrates effectiveness against various backdoor attack strategies (edge-case, distributed, colluding, adaptive) and across different data modalities.
  • Theoretical Guarantees: The paper provides theoretical analysis showing that their approach provably mitigates backdoor attacks and maintains convergence towards benign minima.

Limitations and Future Work

  • Flatness Still Matters: While the defense is robust to flat loss landscapes, extreme flatness can still increase the probability of gradient sign flipping, potentially impacting the defense's effectiveness. The authors suggest that improving gradient estimation accuracy could further enhance performance in such scenarios.
  • Hyperparameter Tuning: While the paper claims the approach is "easy to apply" and doesn't require "expensive hyperparameter tuning," some sensitivity to parameters like the masking threshold ($\tau$) and trim level ($\alpha$) is observed.
  • Future Directions: The authors suggest exploring further combinations with techniques like distributional robust optimization and sharpness-aware minimization to discover more invariant flat minima. They also propose investigating the defense's performance with more diverse and complex backdoor attack scenarios.

Summary updated

Abstract

Federated learning is gaining popularity as it enables training of high-utility models across several clients without directly sharing their private data. As a downside, the federated setting makes the model vulnerable to various adversarial attacks in the presence of malicious clients. Specifically, an adversary can perform backdoor attacks to control model predictions via poisoning the training dataset with a trigger. In this work, we propose a mitigation for backdoor attacks in a federated learning setup. Our solution forces the model optimization trajectory to focus on the invariant directions that are generally useful for utility and avoid selecting directions that favor few and possibly malicious clients. Concretely, we consider the sign consistency of the pseudo-gradient (the client update) as an estimation of the invariance. Following this, our approach performs dimension-wise filtering to remove pseudo-gradient elements with low sign consistency. Then, a robust mean estimator eliminates outliers among the remaining dimensions. Our theoretical analysis further shows the necessity of the defense combination and illustrates how our proposed solution defends the federated learning model. Empirical results on three datasets with different modalities and varying number of clients show that our approach mitigates backdoor attacks with a negligible cost on the model utility.
BibTeX
@inproceedings{wang2023invariant,
  author = {Xiaoyang Wang and Dimitrios Dimitriadis and Oluwasanmi O Koyejo and Shruti Tople},
  title = {{Invariant Aggregator for Defending against Federated Backdoor Attacks}},
  booktitle = {International Conference on Learning Representations},
  year = {2023},
  url = {https://openreview.net/forum?id=3ZHX6_Mydd7},
}
Figure