Instance-wise Batch Label Restoration via Gradients in Federated Learning

Kailang Ma, Yu Sun, Jian Cui, Dawei Li, Zhenyu Guan, Jianwei LiuICLR 2023 · OpenReview

TL;DR

This paper introduces iLRG, an analytical attack that breaks the privacy of Federated Learning by revealing exactly how many instances of each class are present in a training batch. By solving a system of linear equations derived from final-layer gradients, the researchers can recover labels with over 96% accuracy even for massive batch sizes (up to 4096), significantly enhancing the power of subsequent image reconstruction attacks.

What Problem Does This Paper Address?

Federated Learning (FL) is often touted for its privacy-preserving nature: clients only share model updates (gradients) rather than their raw private data. However, "gradient inversion" attacks have shown that it's possible to reconstruct original images from these shared gradients.

A major bottleneck for these attacks is label restoration. If an attacker doesn't know which labels are in a batch, the search space for reconstructing images is massive. Previous methods were limited: they could often tell if a class existed in a batch (class-wise restoration) but failed to determine how many instances of that class were present (instance-wise restoration). Furthermore, many existing methods relied on the assumption that models use non-negative activation functions (like ReLU), making them ineffective against modern architectures using Leaky ReLU or Swish.

Methods

The authors propose iLRG (instance-wise Labels Restoration from Gradients), an analytical method that doesn't require iterative optimization.

Overview of the iLRG method

The process follows three main steps:

  1. Class-wise Embedding Reconstruction: The attacker extracts the average "embedding" (the input to the final classification layer) for each class by calculating the quotient of the weight and bias gradients of the final layer.
  2. Probability Estimation: These reconstructed embeddings are fed back into the final layer to estimate the post-softmax probabilities the model would have produced for those classes.
  3. Solving the Linear System: The core innovation is representing the relationship between gradients, probabilities, and labels as a system of linear equations. The "Number of Instances" (NoI) per class becomes the unknown variable. The authors solve this system using the Moore-Penrose pseudoinverse algorithm to find the most likely count of each label in the batch.

Crucially, this method works even when activations produce negative values and is particularly potent against untrained models, which are common starting points in many FL scenarios.

Datasets

The researchers evaluated their method across three standard image classification benchmarks:

  • MNIST: 10 classes of grayscale handwritten digits (28x28).
  • CIFAR100: 100 classes of color images (32x32).
  • ImageNet (ILSVRC 2012): 1000 classes of high-resolution color images (224x224).

They tested these datasets against various architectures, including a 3-layer FCN, LeNet-5, VGG-16, and ResNet-50.

Key Insights and Ideas

  • The "Untrained" Vulnerability: The attack is most effective at the very beginning of training. This is because untrained models exhibit "intra-class uniformity"—the model hasn't yet learned to distinguish between different instances of the same class, making their embeddings very similar and easier to solve for analytically.
Attack accuracy vs. Model training progression
  • Labels as a Counting Problem: By treating label restoration as a linear algebra problem rather than an optimization problem, the authors made the attack deterministic, extremely fast, and scalable to much larger batch sizes than previously thought possible.
  • Synergy with Reconstruction: The paper demonstrates that knowing the exact label counts acts as a powerful "prior." When combined with existing reconstruction attacks like InvertingGradients (IG), the quality of the recovered images improves drastically.
  • Resilience to Defenses: The authors found that common privacy defenses like Differential Privacy (adding noise) and Gradient Sparsification (pruning small gradients) have surprisingly little impact on the accuracy of iLRG when targeting untrained models.

Key Results

  • High Accuracy: The method achieved over 99% Label existence Accuracy (LeAcc) and exceeded 96% Label number Accuracy (LnAcc) in most experimental setups.
  • Massive Scalability: While prior attacks struggled with batch sizes over 64, iLRG remained effective even with a batch size of 4096.
  • Visual Improvement: Using iLRG to provide labels for image reconstruction attacks resulted in a 6-7 dB increase in PSNR (Peak Signal-to-Noise Ratio) on MNIST and CIFAR100, meaning the reconstructed images were significantly clearer and more recognizable.
  • Activation Agnostic: Unlike previous work, iLRG successfully handled models using Leaky ReLU and Swish activations.

Limitations and Future Work

  • The "Well-Trained" Shield: The attack's performance degrades as the model becomes better trained. As the model learns to see the unique features of individual images (reducing intra-class uniformity), the analytical approximations used in iLRG become less accurate.
  • Extreme Imbalance: While the method handles some imbalance, extremely imbalanced batches (where one class dominates almost the entire batch) can still lead to entanglement errors in the gradients.
  • Future Defense: The authors suggest that the only robust way to prevent such attacks is to avoid exchanging precise gradients altogether, perhaps through more aggressive quantization or secure multi-party computation, though these often come with a trade-off in model accuracy.

Summary updated

BibTeX
@inproceedings{ma2023instance,
  author = {Kailang Ma and Yu Sun and Jian Cui and Dawei Li and Zhenyu Guan and Jianwei Liu},
  title = {{Instance-wise Batch Label Restoration via Gradients in Federated Learning}},
  booktitle = {International Conference on Learning Representations},
  year = {2023},
  url = {https://openreview.net/forum?id=YIOkwShXJ1},
}
Figure