Abstract
Despite the potential of federated learning (FL), it remains vulnerable to backdoor attacks. While many robust aggregation methods have been proposed, they are primarily validated in Computer Vision (CV). This paper demonstrates that NLP backdoors are significantly harder to defend against than those in CV. We provide a theoretical analysis showing that malicious update detection error probabilities are determined by relative backdoor strengths, which are typically smaller in NLP. To address this, we propose Dim-Krum, a novel federated aggregation algorithm that focuses on dimensions with higher backdoor strengths. Experimental results across multiple NLP benchmarks validate its effectiveness in suppressing backdoors while maintaining high model accuracy.
1. Introduction
Federated Learning (FL) enables collaborative model training across decentralized clients without requiring direct data sharing. However, this privacy-preserving nature introduces a critical security risk: backdoor attacks. In a backdoor attack, a malicious client poisons the global model such that it behaves normally on clean data but produces specific, incorrect outputs when triggered by a particular pattern (e.g., a specific word or phrase in a sentiment analysis task).
While Byzantine-robust aggregation methods like Krum were designed to mitigate these risks, they often fail in NLP contexts. This paper investigates why NLP backdoors are more stealthy than their CV counterparts and introduces Dim-Krum to bridge this defensive gap.
2. The Challenge: Why NLP Backdoors are Stealthier
The authors identify a fundamental difference between CV and NLP backdoor signals. In CV, malicious updates often affect a broad range of parameters. In contrast, NLP backdoors—such as those targeting specific word embeddings or syntactic structures—are often "spiky" and sparse.
Theoretical Motivation
The paper provides a theoretical analysis (Theorem 1) suggesting that the probability of failing to detect a malicious update is inversely related to the relative backdoor strength. In NLP, because the attack signal is often concentrated in a tiny fraction of the model's dimensions, the global average of updates "drowns out" the malicious signal with the noise of honest updates. This makes traditional distance-based defenses like Krum ineffective, as the malicious update appears statistically similar to honest updates when viewed across the entire parameter space.
3. The Dim-Krum Algorithm
To counter this stealth, Dim-Krum shifts the focus from global model updates to specific, high-signal dimensions. The algorithm consists of four primary components:
- Dimension-wise Selection: Instead of calculating distances across all parameters, Dim-Krum identifies a small fraction $\rho$ (e.g., 0.1%) of dimensions that exhibit the largest changes. These dimensions are the most likely candidates for containing concentrated "poison."
- Krum-based Filtering: The algorithm performs outlier detection using the Krum framework, but only within these selected high-signal dimensions. This amplifies the signal-to-noise ratio, making malicious clients stand out.
- Memory Mechanism: To ensure temporal consistency, Dim-Krum employs an exponential moving average (EMA) to track client behavior over multiple rounds. This prevents attackers from intermittently bypassing detection.
- Adaptive Noise: A calculated amount of noise is added to the final aggregated update to neutralize any residual backdoor signals that might have survived the filtering process.
4. Experimental Evaluation
The researchers evaluated Dim-Krum against several baselines, including FedAvg, Median, FoolsGold, and standard Krum. Testing was conducted on four NLP classification benchmarks:
- SST-2: Sentiment analysis of movie reviews.
- IMDb: Large-scale movie review sentiment.
- Amazon: Product review classification.
- AgNews: Four-category news topic classification.
The defense was tested against four sophisticated attack vectors: Embedding Poisoning (EP), BadWord (trigger words), BadSent (sentence-level triggers), and HiddenKiller (syntactic triggers).
5. Key Results
Dim-Krum demonstrated superior performance in neutralizing attacks while preserving utility:
- Backdoor Suppression: The average Backdoor Attack Success Rate (ASR) plummeted from 94.35% in standard FedAvg and 53.69% in Krum to just 18.39% with Dim-Krum.
- Resilience to Complex Attacks: In the "BadSent" scenario, where standard Krum was almost entirely ineffective (97.45% ASR), Dim-Krum reduced the ASR to 22.16%.
- Utility Preservation: The "Clean Accuracy" (performance on legitimate data) remained robust, typically seeing only a marginal drop of 1-2% compared to the non-robust FedAvg baseline.
- Robustness to Non-IID Data: The method remained effective even when client data distributions were highly heterogeneous (Non-IID) and when up to 40% of the clients were malicious.
6. Conclusion
Dim-Krum addresses a critical vulnerability in federated NLP by recognizing that malicious signals are often dimension-specific. By isolating and analyzing these "spiky" updates, the algorithm provides a robust defense where traditional methods fail. While future work may explore adaptive attackers who attempt to distribute their signal to evade Top-K filters, Dim-Krum currently represents a significant step toward secure federated learning for natural language processing.
Summary updated