Boosting Gradient Leakage Attacks: Data Reconstruction in Realistic FL Settings

Mingyuan Fan, Fuyi Wang, Cen Chen, Jianying Zhou

Introduction

Federated Learning (FL) has emerged as a cornerstone of privacy-preserving machine learning, enabling multiple parties to collaboratively train models without ever sharing their raw data. By only exchanging model updates—or gradients—FL aims to keep sensitive information localized. However, the security of this paradigm rests on the assumption that gradients do not leak significant information about the underlying training data.

Recent research into Gradient Leakage Attacks (GLAs) has challenged this assumption, demonstrating that it is theoretically possible to reverse-engineer gradients to reconstruct original images. Despite these findings, many practitioners have remained skeptical, viewing GLAs as a "toy" problem that only manifests in unrealistic scenarios involving tiny batch sizes or overly simplistic models. This paper introduces FEDLEAK, a novel attack framework that bridges the gap between theory and practice, proving that high-fidelity data reconstruction is possible even in complex, large-scale, and realistic FL environments.

The Optimization Challenge

Previous GLAs often failed in realistic settings not because the gradients lacked information, but because the optimization process used to reconstruct the images was flawed. Reconstructing data from gradients is essentially an inverse problem where an attacker seeks a "dummy" image that produces a gradient identical to the one observed. In complex models or large batches, the optimization landscape becomes highly non-convex and "craggy," causing standard attacks to get stuck in poor local minima.

FEDLEAK addresses this by identifying that the solution to the reconstruction problem is usually unique, but the path to finding it is obscured by noise. The authors propose two primary innovations to stabilize and enhance this optimization process:

1. Partial Gradient Matching

Instead of attempting to match every single element of a gradient vector—which can contain millions of parameters—FEDLEAK focuses exclusively on the top $R\%$ of gradient elements with the largest magnitudes. Mathematically, these high-magnitude elements carry the most significant "signal" regarding the input data. By ignoring the smaller elements, which often act as noise during the reconstruction phase, FEDLEAK improves the curvature of the optimization problem, making it easier for the algorithm to converge on the correct image.

2. Hessian-Free Gradient Regularization

To further stabilize reconstruction, the authors incorporate second-order information (the Hessian matrix). Since calculating a full Hessian is computationally prohibitive for modern deep neural networks, FEDLEAK utilizes a clever approximation based on a Taylor expansion. This "Hessian-free" approach creates an adaptive step-size mechanism that prevents the attack from overshooting the solution, leading to smoother and more consistent convergence.

Experimental Evaluation

The study rigorously tests FEDLEAK across a variety of challenging environments, moving beyond standard benchmarks like MNIST and CIFAR-10 to include:

  • High-Resolution Data: ImageNet.
  • Medical Datasets: HAM10000 (skin lesion images) and Lung-Colon Cancer histopathological images. These are critical use cases, as FL is frequently proposed for healthcare sectors where data privacy is a legal requirement.

Key Results

  • Superior Fidelity: On the CIFAR-10 dataset with a batch size of 16, FEDLEAK achieved a Peak Signal-to-Noise Ratio (PSNR) of 23.06. In comparison, state-of-the-art baselines like InvertingGrad and GradInversion struggled to exceed a PSNR of 11, resulting in images that were essentially unrecognizable noise.
  • Scaling to Large Batches: One of the most significant findings is FEDLEAK's ability to scale. While most attacks fail as the batch size increases, FEDLEAK maintained high-fidelity reconstruction at a batch size of 64—a common configuration in industrial FL.
  • Resilience to Defenses: The attack proved remarkably resilient against common mitigation strategies, including Gradient Pruning and Differential Privacy (DP). While aggressive DP can stop the attack, it often does so at the cost of the model's primary task accuracy, creating a difficult trade-off for practitioners.
  • Semantic Recovery in Healthcare: In medical datasets, FEDLEAK achieved over a 20% Attack Success Rate, where human volunteers could successfully identify key semantic features in the reconstructed images, posing a direct threat to patient privacy.

Discussion and Future Work

FEDLEAK demonstrates that the "privacy by design" nature of Federated Learning may be more fragile than previously thought. The research highlights that the most significant leaks occur through high-magnitude gradient elements.

However, the attack is not without limitations. FEDLEAK is computationally more intensive than its predecessors, requiring approximately 20 minutes to reconstruct a batch on an NVIDIA RTX 4090. Furthermore, while it bypasses many heuristic defenses, cryptography-based methods like Secure Aggregation remain effective at blocking these attacks, albeit with significant communication overhead.

Future research should focus on developing targeted defenses that specifically obfuscate the high-magnitude gradient elements identified by FEDLEAK, potentially offering a more efficient way to secure FL without the heavy performance penalties of current differential privacy or cryptographic methods.

Summary updated

BibTeX
@inproceedings{fan2025boosting,
  author = {Mingyuan Fan and Fuyi Wang and Cen Chen and Jianying Zhou},
  title = {{Boosting Gradient Leakage Attacks: Data Reconstruction in Realistic FL Settings}},
  booktitle = {USENIX Security},
  year = {2025},
  url = {https://openreview.net/forum?id=pSeZF4NTSs},
}
Figure