Invariant Aggregator for Defending Federated Backdoor Attacks

Xiaoyang Wang, Dimitrios Dimitriadis, Sanmi Koyejo, Shruti Tople

Abstract

Federated Learning (FL) enables collaborative model training while maintaining data privacy, but its decentralized nature introduces significant security risks, specifically backdoor attacks. Traditional defenses often fail in modern neural networks characterized by flat loss landscapes, where malicious updates can easily blend with benign noise. This paper introduces the Invariant Aggregator, a novel defense mechanism that identifies "invariant directions"—gradient dimensions where a majority of clients agree on the direction of improvement. By applying a dimension-wise AND-mask followed by a trimmed-mean estimator, the proposed method effectively filters out stealthy backdoor triggers. Experimental results across visual, textual, and tabular datasets demonstrate a 61.6% reduction in attack success rate with minimal impact on primary task accuracy.

Introduction: The Challenge of Flat Landscapes

In Federated Learning, a central server aggregates updates from multiple clients to train a global model. Backdoor attacks exploit this by injecting malicious "triggers" (like specific pixel patterns or keywords) into the training data of a subset of clients. The goal is to compromise the model's behavior on specific inputs while maintaining normal performance on clean data.

A critical, often overlooked vulnerability in FL is the flat loss landscape typical of large-scale models like ResNets. In these regions, the gradient of the loss function is small, making it difficult to distinguish between legitimate updates and malicious perturbations. Existing defenses that rely on vector-wise similarity or simple robust statistics struggle because the malicious signal can hide within the variance of a flat landscape.

Comparison of loss landscapes and gradient updates for benign and malicious clients.

Methodology: The Invariant Aggregator

The Invariant Aggregator addresses these vulnerabilities by operating dimension-wise on model updates (pseudo-gradients) using a two-step process:

  1. AND-mask (Invariant Direction Identification): The server evaluates the "sign consistency" for every parameter across all reporting clients. If the consensus on whether a parameter should increase or decrease falls below a predefined threshold $\tau$, that dimension is masked to zero. This ensures that only directions beneficial to the collective are updated, preventing small groups of attackers from steering the model toward a backdoor.
  2. Trimmed-Mean Estimator: To handle potential outliers in dimensions with high sign consistency, the server applies a trimmed-mean. This involves discarding the most extreme values (highest and lowest) before calculating the average, further neutralizing the influence of malicious clients.

The core innovation lies in the shift from magnitude-based filtering to sign-based consistency. While gradient magnitudes are unreliable in flat regions, the direction (sign) remains a robust indicator of an update's general utility.

Key Insights and Theoretical Foundations

The paper highlights that flatness, while beneficial for generalization, creates a "security trap." Attackers can mimic benign behavior with minimal penalty, making their updates nearly indistinguishable from legitimate ones when viewed holistically.

Visualizing the failure of median-based aggregation in the presence of malicious outliers.

The authors provide theoretical evidence that neither the AND-mask nor the Trimmed-Mean is sufficient in isolation. The AND-mask filters out non-invariant malicious directions, while the Trimmed-Mean prevents "value-scaling" attacks in invariant directions. Together, they provide a provable mitigation strategy that maintains convergence toward the benign global minimum.

Experimental Evaluation

The Invariant Aggregator was tested against twelve state-of-the-art defenses across three diverse modalities:

  • Visual (CIFAR-10): ResNet-18 for object detection.
  • Textual (Twitter): Two-layer LSTM for sentiment analysis.
  • Tabular (Phishing): Fully connected network for fraud detection.

Results

  • Attack Mitigation: The defense reduced the backdoor Attack Success Rate (ASR) by an average of 61.6%.
  • Utility Preservation: The method is highly surgical, resulting in only a 1.2% drop in benign accuracy compared to standard FedAvg.
  • Robustness: It proved effective against sophisticated "Edge-case" and "Adaptive" attacks designed to bypass traditional masks.

Conclusion and Future Directions

The Invariant Aggregator offers a robust solution for securing Federated Learning in the presence of flat loss landscapes. While highly effective, the defense showed slightly lower performance on text data, likely due to lower natural sign consistency in sentiment analysis tasks. Future research may explore integrating this approach with Sharpness-Aware Minimization (SAM) to proactively identify robust, invariant minima that are inherently more resistant to poisoning.

Summary updated

Abstract

Federated learning is gaining popularity as it enables training high-utility models across several clients without directly sharing their private data. As a downside, the federated setting makes the model vulnerable to various adversarial attacks in the presence of malicious clients. Despite the theoretical and empirical success in defending against attacks that aim to degrade models' utility, defense against backdoor attacks that increase model accuracy on backdoor samples exclusively without hurting the utility on other samples remains challenging. To this end, we first analyze the vulnerability of federated learning to backdoor attacks over a flat loss landscape which is common for well-designed neural networks such as Resnet [He et al., 2015] but is often overlooked by previous works. Over a flat loss landscape, misleading federated learning models to exclusively benefit malicious clients with backdoor samples do not require a significant difference between malicious and benign client-wise updates, making existing defenses insufficient. In contrast, we propose an invariant aggregator that redirects the aggregated update to invariant directions that are generally useful via selectively masking out the gradient elements that favor few and possibly malicious clients regardless of the difference magnitude. Theoretical results suggest that our approach provably mitigates backdoor attacks over both flat and sharp loss landscapes. Empirical results on three datasets with different modalities and varying numbers of clients further demonstrate that our approach mitigates a broad class of backdoor attacks with a negligible cost on the model utility.
BibTeX
@inproceedings{wang2023invariant,
  author = {Xiaoyang Wang and Dimitrios Dimitriadis and Sanmi Koyejo and Shruti Tople},
  title = {{Invariant Aggregator for Defending Federated Backdoor Attacks}},
  booktitle = {International Conference on Learning Representations},
  year = {2023},
  url = {https://openreview.net/forum?id=3ZHX6_Mydd7},
}
Figure