Inverting Gradients - How easy is it to break privacy in federated learning?

Jonas Geiping, Hartmut Bauermeister, Hannah Dröge, Michael Moeller

Federated learning is widely adopted and marketed as a "privacy-preserving" approach to machine learning because it shares model gradients instead of raw user data. However, this paper demonstrates that this protection is surprisingly fragile. By optimizing for gradient direction rather than magnitude, researchers can faithfully reconstruct high-resolution images from gradients, even in deep, trained models and across large batches.

The False Security of Gradient Sharing

In Federated Learning (FL), multiple users collaboratively train a model by sending parameter updates (gradients) to a central server. The prevailing wisdom has been that gradients are "safe" to share because they represent a condensed, aggregated update rather than the raw data itself.

High-resolution image reconstruction from gradients.

While previous research (like Deep Leakage from Gradients, or DLG) showed that data could be recovered, those attacks were limited to contrived settings: shallow networks, untrained weights, or very small images. The authors of this paper ask a critical question: Is federated learning actually secure in realistic, industrial-scale settings? They specifically target the "honest-but-curious" server—a server that follows the protocol perfectly but attempts to "spy" on user data using the information it receives.

A New Approach to Gradient Inversion

The authors introduce a robust numerical method for "inverting" gradients that succeeds where previous methods failed. Their approach differs from prior work in three key ways:

  1. Cosine Similarity over Euclidean Loss: Previous methods tried to match the magnitude of gradients using Euclidean distance. This paper argues that magnitude mostly reflects how well the model is already trained. Instead, they minimize a loss based on the angle (cosine similarity) between a dummy gradient and the observed gradient. This focuses on the direction of the gradient, which carries far more information about the input features.
  2. Adversarial Optimization: Instead of standard second-order solvers like L-BFGS, they use Adam with signed gradients. Borrowed from the world of adversarial attacks, this technique makes optimization more stable when dealing with the non-smooth nature of deep neural networks (like those using ReLU activations).
  3. Total Variation (TV) Prior: They incorporate a TV regularizer, which encourages the reconstructed image to have the spatial smoothness typical of natural photos, effectively filtering out "salt-and-pepper" noise.

Additionally, the paper provides a mathematical proof showing that the input to any Fully Connected (FC) layer can be reconstructed analytically, regardless of the rest of the architecture.

Comparison of reconstruction methods on untrained and trained ResNets. The proposed method succeeds where previous Euclidean-based methods fail.

Key Insights and Results

The researchers evaluated their attack on standard computer vision benchmarks, including CIFAR-10/100 and high-resolution ImageNet (ILSVRC2012) images, revealing several critical vulnerabilities in FL:

  • Direction is Everything: The core insight is that the direction of a gradient tells you what the data looks like, while the magnitude only tells you how much the model needs to learn from it. Ignoring magnitude makes the attack significantly more potent.
  • Deep Networks Offer No "Defense-in-Depth": One might assume a 152-layer ResNet would "hide" the input better than a 4-layer CNN. The authors show the opposite: deep networks are just as vulnerable, if not more so, because they contain so many parameters that can be used to "solve" for the input. The authors successfully reconstructed ImageNet-sized images from a trained ResNet-152, a feat previously thought impossible.
  • Quantitative Superiority: On a trained ResNet-20, the proposed method achieved a Peak Signal-to-Noise Ratio (PSNR) of ~14 dB, compared to just ~7 dB for the previous state-of-the-art Euclidean approach (which essentially failed to produce a recognizable image, as seen in the comparison figure).
  • The "Class Bias" of Trained Models: When attacking a well-trained model, reconstructions tend to look like "prototypical" versions of the class. The model’s internal knowledge of a category actually assists the attacker in reconstructing the private data.
  • Aggregation is Not Enough: A common defense is to average gradients from a batch of images. The authors show that even in a batch of 100 images, several individual images remain clearly recognizable. The attack also remained effective even when the user performed multiple local training steps (Federated Averaging).

Limitations and The Path Forward

While the attack is computationally expensive—requiring roughly twice the effort of a standard training step per iteration—this is a negligible barrier for a powerful central server. Data augmentation (like random cropping) can make it harder to determine the exact original position of objects, but the core content is still recovered.

The authors conclude that privacy is not an "innate" property of federated learning. To truly guarantee user privacy, systems may need to rely on Provable Differential Privacy (adding calibrated noise to gradients), though this often comes at the cost of model accuracy.

Summary updated

Abstract

The idea of federated learning is to collaboratively train a neural network on a server. Each user receives the current weights of the network and in turns sends parameter updates (gradients) based on local data. This protocol has been designed not only to train neural networks data-efficiently, but also to provide privacy benefits for users, as their input data remains on device and only parameter gradients are shared. But how secure is sharing parameter gradients? Previous attacks have provided a false sense of security, by succeeding only in contrived settings - even for a single image. However, by exploiting a magnitude-invariant loss along with optimization strategies based on adversarial attacks, we show that is is actually possible to faithfully reconstruct images at high resolution from the knowledge of their parameter gradients, and demonstrate that such a break of privacy is possible even for trained deep networks. We analyze the effects of architecture as well as parameters on the difficulty of reconstructing an input image and prove that any input to a fully connected layer can be reconstructed analytically independent of the remaining architecture. Finally we discuss settings encountered in practice and show that even averaging gradients over several iterations or several images does not protect the user's privacy in federated learning applications.
BibTeX
@inproceedings{geiping2020inverting,
  author = {Jonas Geiping and Hartmut Bauermeister and Hannah Dröge and Michael Moeller},
  title = {{Inverting Gradients - How easy is it to break privacy in federated learning?}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2020},
  url = {https://openreview.net/forum?id=14UOfLLkes},
}
Figure