Evaluating Selective Encryption Against Gradient Inversion Attacks

Jiajun Gu, Yuhang Yao, Shuaiqi Wang, Carlee Joe-WongarXiv 2025 · DOI · arXiv

Abstract

Distributed learning frameworks, such as Federated Learning, aim to preserve privacy by sharing model gradients instead of raw data. However, gradient inversion attacks can mathematically reverse-engineer these shared gradients to reconstruct sensitive training data. While encryption offers a robust defense, encrypting all gradients is computationally prohibitive. This paper presents a systematic evaluation of selective encryption, proposing a theoretical framework to identify "significant" gradient elements. The study reveals that simple gradient magnitude (Grad) is a highly effective and efficient metric for protecting private data across various models and tasks.

Introduction: The Threat of Gradient Inversion

In distributed training, clients compute and share gradients to update a global model. Although raw data remains local, gradient inversion attacks can reconstruct high-fidelity training images or text from these updates. To mitigate this risk without the 100x communication overhead associated with full Homomorphic Encryption, selective encryption identifies and protects only the most critical gradient elements. This research addresses the gap in defining "significance" for privacy preservation by evaluating different metrics against state-of-the-art attacks.

Methodology and Significance Metrics

The authors formalize selective encryption using a binary mask $m$ that selects the top $s$ gradient elements based on a significance metric. They introduce a distance-based significance analysis framework designed to maximize the mathematical distance between an attacker's reconstruction and the ground truth data.

Overview of the selective encryption pipeline against gradient inversion attacks.

The study evaluates four primary metrics:

  1. Sensitivity (Sens): Based on second-order derivatives; traditionally used in explainability but computationally intensive.
  2. Product Significance (ProdSig): A novel metric that multiplies gradient magnitude by model parameter magnitude.
  3. Gradient Magnitude (Grad): Selecting elements with the largest absolute values; computationally "free" as gradients are already computed.
  4. Parameter Magnitude (Param): Selecting gradients corresponding to the largest model weights.

These defenses were tested against three major attacks: Inverting Gradients (IG) for images (CIFAR-100) and LAMP and DAGER for text (CoLA, SST-2, Rotten Tomatoes) using models like LeNet, CNN, BERT, and GPT-2.

Key Results and Performance Analysis

The evaluation demonstrates that no single metric is a universal solution, but certain patterns emerge. Grad consistently performs well against optimization-based attacks (IG and LAMP) and is highly efficient. In contrast, Sens is often too slow for practical use in large-scale models.

Performance of significance metrics against various gradient inversion attacks. Higher star counts indicate better protection.

Key findings include:

  • Efficiency of Grad: The Grad metric achieved target protection levels (LPIPS $\geq$ 0.3 for images, ROUGE-1 $\leq$ 0.2 for text) with only 10% to 40% encryption.
  • Computational Savings: For BERT, calculating Sens was infeasible, while ProdSig took 0.1 seconds and Grad added zero overhead.
  • Attack Specificity: While Param fails against optimization attacks, it is exceptionally effective at stopping analytical attacks like DAGER.

Discussion and Future Directions

The research highlights that protecting specific layers, such as attention layers in GPT-2, is insufficient as sensitive information can still leak through other network components. Future work should explore adaptive defense strategies that dynamically adjust encryption ratios based on real-time threat levels or specific model architectures. Additionally, hybrid metrics combining Grad and Param could offer more robust protection against both optimization and analytical attack vectors. While this study focused on the worst-case scenario of a batch size of 1, investigating the interaction between batch size and selective encryption remains a promising area for further exploration.

Summary updated

Abstract

Gradient inversion attacks pose significant privacy threats to distributed training frameworks such as federated learning, enabling malicious parties to reconstruct sensitive local training data from gradient communications between clients and an aggregation server during the aggregation process. While traditional encryption-based defenses, such as homomorphic encryption, offer strong privacy guarantees without compromising model utility, they often incur prohibitive computational overheads. To mitigate this, selective encryption has emerged as a promising approach, encrypting only a subset of gradient data based on the data's significance under a certain metric. However, there have been few systematic studies on how to specify this metric in practice. This paper systematically evaluates selective encryption methods with different significance metrics against state-of-the-art attacks. Our findings demonstrate the feasibility of selective encryption in reducing computational overhead while maintaining resilience against attacks. We propose a distance-based significance analysis framework that provides theoretical foundations for selecting critical gradient elements for encryption. Through extensive experiments on different model architectures (LeNet, CNN, BERT, GPT-2) and attack types, we identify gradient magnitude as a generally effective metric for protection against optimization-based gradient inversions. However, we also observe that no single selective encryption strategy is universally optimal across all attack scenarios, and we provide guidelines for choosing appropriate strategies for different model architectures and privacy requirements.
BibTeX
@misc{gu2025evaluating,
  author = {Jiajun Gu and Yuhang Yao and Shuaiqi Wang and Carlee Joe-Wong},
  title = {{Evaluating Selective Encryption Against Gradient Inversion Attacks}},
  year = {2025},
  eprint = {2508.04155},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2508.04155},
}
Figure