Exploit Gradient Skewness to Circumvent Byzantine Defenses for Federated Learning

Yuchen Liu, Chen Chen 0043, Lingjuan Lyu, Yaochu Jin, Gang Chen 0001

Introduction

Federated Learning (FL) enables collaborative model training across distributed clients while maintaining data privacy. However, its decentralized nature introduces significant vulnerability to Byzantine attacks, where malicious participants submit poisoned gradients to degrade model performance. To counter this, current defenses rely on Robust Aggregation Rules (AGRs) that operate on a common inductive bias: they assume honest gradients are densely clustered, while malicious updates appear as outliers.

This paper identifies a fundamental flaw in this density-based logic. In real-world scenarios characterized by non-IID (heterogeneous) data, honest gradients are often naturally skewed. This skewness allows for a new class of attack, STRIKE (gRadIent sKEw attack), which hides malicious updates within these skewed clusters, tricking defenses into discarding the true optimal gradient.

The Gradient Skewness Problem

Traditional robust aggregators assume that the majority of honest gradients will be concentrated around the optimal update. However, data heterogeneity causes honest gradients to form lopsided distributions. As shown in Figure 1, a large group of honest gradients can cluster far from the actual average (the optimal gradient). Because current defenses prioritize the densest regions, they inadvertently favor these skewed, sub-optimal directions—or can be manipulated into selecting malicious ones.

Visualization of gradient skewness where the optimal gradient is an outlier relative to the majority cluster.

Methodology: The STRIKE Attack

The authors propose STRIKE, a two-stage attack designed to exploit this inherent skewness:

  1. Search for Skewed Honest Gradients: Utilizing Karl Pearson’s formula relating mean, median, and mode, the attacker identifies the direction of skewness. By projecting honest gradients onto a search vector ($u_{search} = g_{med} - \bar{g}$), the attacker locates the most densely skewed cluster.
  2. Construct Byzantine Gradients: The attacker then solves a constrained optimization problem to place malicious gradients within the diameter of the identified skewed cluster. This ensures the malicious updates are "covert"—hidden within the density—while maximizing their distance from the true optimal gradient to inflict maximum damage.

This strategy effectively tricks the server into treating the skewed/malicious cluster as the "honest" majority, while the actual average of all honest clients is discarded as an outlier.

Experimental Evaluation

The effectiveness of STRIKE was evaluated on three benchmarks: CIFAR-10, ImageNet-12, and FEMNIST. These datasets were partitioned in a non-IID manner to simulate realistic federated environments. The attack was tested against seven state-of-the-art robust defenses, including Multi-Krum, Median, RFA, Aksel, CClip, DnC, and RBTM.

Key Results

STRIKE consistently outperforms existing Byzantine attacks such as BitFlip, LIE, and Min-Max. As detailed in the performance comparison below, STRIKE leads to significantly lower model test accuracies compared to baseline attacks across all tested defenses.

Comparison of model test accuracy under various Byzantine attacks across different robust aggregation rules.

Notable findings include:

  • Significant Accuracy Degradation: On the FEMNIST dataset against the DnC defense, STRIKE increased attack effectiveness by 57.84% over the best baseline.
  • Impact on Large-Scale Data: Against CClip on ImageNet-12, STRIKE reduced model accuracy to 8.33%, whereas other attacks failed to drop it below 11-20%.
  • Sensitivity to Heterogeneity: The attack's power scales with data heterogeneity; as the Dirichlet concentration decreases (increasing non-IIDness), the gradient skew becomes more pronounced, making STRIKE more effective.

Conclusion

The success of STRIKE demonstrates that relying solely on gradient density for robustness is insufficient in non-IID federated learning. Future defenses must move beyond simple spatial clustering and become "skew-aware," accounting for the natural distribution shapes of heterogeneous data to effectively mitigate such sophisticated Byzantine threats.

Summary updated

Abstract

Federated Learning (FL) is notorious for its vulnerability to Byzantine attacks. Most current Byzantine defenses share a common inductive bias: among all the gradients, the densely distributed ones are more likely to be honest. However, such a bias is a poison to Byzantine robustness due to a newly discovered phenomenon in this paper - gradient skew. We discover that a group of densely distributed honest gradients skew away from the optimal gradient (the average of honest gradients) due to heterogeneous data. This gradient skew phenomenon allows Byzantine gradients to hide within the densely distributed skewed gradients. As a result, Byzantine defenses are confused into believing that Byzantine gradients are honest. Motivated by this observation, we propose a novel skew-aware attack called STRIKE: first, we search for the skewed gradients; then, we construct Byzantine gradients within the skewed gradients. Experiments on three benchmark datasets validate the effectiveness of our attack
BibTeX
@inproceedings{liu2025exploit,
  author = {Yuchen Liu and Chen Chen and Lingjuan Lyu and Yaochu Jin and Gang Chen},
  title = {{Exploit Gradient Skewness to Circumvent Byzantine Defenses for Federated Learning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i18.34094},
}
Figure