Introduction
Federated Learning (FL) enables collaborative model training across distributed clients while maintaining data privacy. However, its decentralized nature introduces significant vulnerability to Byzantine attacks, where malicious participants submit poisoned gradients to degrade model performance. To counter this, current defenses rely on Robust Aggregation Rules (AGRs) that operate on a common inductive bias: they assume honest gradients are densely clustered, while malicious updates appear as outliers.
This paper identifies a fundamental flaw in this density-based logic. In real-world scenarios characterized by non-IID (heterogeneous) data, honest gradients are often naturally skewed. This skewness allows for a new class of attack, STRIKE (gRadIent sKEw attack), which hides malicious updates within these skewed clusters, tricking defenses into discarding the true optimal gradient.
The Gradient Skewness Problem
Traditional robust aggregators assume that the majority of honest gradients will be concentrated around the optimal update. However, data heterogeneity causes honest gradients to form lopsided distributions. As shown in Figure 1, a large group of honest gradients can cluster far from the actual average (the optimal gradient). Because current defenses prioritize the densest regions, they inadvertently favor these skewed, sub-optimal directions—or can be manipulated into selecting malicious ones.
Methodology: The STRIKE Attack
The authors propose STRIKE, a two-stage attack designed to exploit this inherent skewness:
- Search for Skewed Honest Gradients: Utilizing Karl Pearson’s formula relating mean, median, and mode, the attacker identifies the direction of skewness. By projecting honest gradients onto a search vector ($u_{search} = g_{med} - \bar{g}$), the attacker locates the most densely skewed cluster.
- Construct Byzantine Gradients: The attacker then solves a constrained optimization problem to place malicious gradients within the diameter of the identified skewed cluster. This ensures the malicious updates are "covert"—hidden within the density—while maximizing their distance from the true optimal gradient to inflict maximum damage.
This strategy effectively tricks the server into treating the skewed/malicious cluster as the "honest" majority, while the actual average of all honest clients is discarded as an outlier.
Experimental Evaluation
The effectiveness of STRIKE was evaluated on three benchmarks: CIFAR-10, ImageNet-12, and FEMNIST. These datasets were partitioned in a non-IID manner to simulate realistic federated environments. The attack was tested against seven state-of-the-art robust defenses, including Multi-Krum, Median, RFA, Aksel, CClip, DnC, and RBTM.
Key Results
STRIKE consistently outperforms existing Byzantine attacks such as BitFlip, LIE, and Min-Max. As detailed in the performance comparison below, STRIKE leads to significantly lower model test accuracies compared to baseline attacks across all tested defenses.
Notable findings include:
- Significant Accuracy Degradation: On the FEMNIST dataset against the DnC defense, STRIKE increased attack effectiveness by 57.84% over the best baseline.
- Impact on Large-Scale Data: Against CClip on ImageNet-12, STRIKE reduced model accuracy to 8.33%, whereas other attacks failed to drop it below 11-20%.
- Sensitivity to Heterogeneity: The attack's power scales with data heterogeneity; as the Dirichlet concentration decreases (increasing non-IIDness), the gradient skew becomes more pronounced, making STRIKE more effective.
Conclusion
The success of STRIKE demonstrates that relying solely on gradient density for robustness is insufficient in non-IID federated learning. Future defenses must move beyond simple spatial clustering and become "skew-aware," accounting for the natural distribution shapes of heterogeneous data to effectively mitigate such sophisticated Byzantine threats.
Summary updated