Quantifying Information Disclosure During Gradient Descent Using Gradient Uniqueness

Sleem Abdelghafar, Maryam Aliakbarpour, Chris JermaineOpenReview 2026 · Submission to ICLR 2026 · Rejected · OpenReview

TL;DR

This paper introduces Gradient Uniqueness (GNQ), a mathematically grounded metric that quantifies how much private information a machine learning model "leaks" about its training data. By identifying and removing data points with high GNQ scores—essentially the "outliers" in gradient space—researchers can achieve privacy protection comparable to Differential Privacy (DP) but with significantly higher model accuracy.

What Problem Does This Paper Address?

The gold standard for data privacy in machine learning is Differential Privacy (DP). While theoretically robust, DP is notoriously "pessimistic." It assumes a worst-case scenario, requiring algorithms like DP-SGD to add noise and clip gradients for every data point during training.

This "one-size-fits-all" approach often destroys model utility. For example, the authors note that a state-of-the-art DP-trained Large Language Model (VaultGemma) only performs on par with the much older GPT-2. Many organizations are unwilling to sacrifice this much performance. This paper seeks a middle ground: a way to rigorously quantify and mitigate privacy risk for a specific dataset and model without the heavy-handed overhead of standard DP.

Methods

The authors propose Gradient Uniqueness (GNQ). At its core, GNQ measures how "unique" a specific data point's gradient is compared to the rest of the training batch.

Mathematically, for a data point $j$ at training step $i$, GNQ is defined as: $$GNQ_{ij} = g_{ij}^\top S^+ g_{ij}$$ Where $g_{ij}$ is the gradient for that point and $S^+$ is the Moore-Penrose pseudoinverse of the sum of the outer products of all other gradients in the batch.

The Intuition: Imagine gradients as vectors in a high-dimensional space. If most vectors point in a similar direction, they form a dense "cloud." If one vector (a data point) points in a completely different direction or is much longer, it stands out. GNQ captures this "outlier-ness." The authors prove that the information an attacker can gain about a point’s membership in the training set is upper-bounded by a function of its GNQ.

The Defense: Instead of adding noise to everything (like DP-SGD), the authors propose a simple "filter and retrain" strategy:

  1. Train a small "proxy" model to calculate GNQ scores for the dataset.
  2. Rank points by their GNQ risk.
  3. Remove the top $p\%$ highest-risk points.
  4. Retrain the full model on the "safe" subset.

Datasets

The authors evaluated their method across diverse domains using:

  • Vision: MNIST, CIFAR-10, CIFAR-100, Tiny ImageNet, and the AT&T Database of Faces.
  • Text: IMDB (sentiment analysis).
  • Models: MLPs, basic CNNs, ResNets, and BERT-based classifiers.

Key Insights and Ideas

  • Privacy is an Outlier Problem: The "aha moment" here is that privacy leakage isn't uniform. Most data points are "safe" because their influence is buried in the average. Only a small fraction of "unique" points—those that force the model to change in a highly specific way—are at high risk of being "memorized" or reconstructed by an attacker.
  • GNQ Predicts Attack Success: The researchers found a near-perfect correlation between a point's GNQ score and the success rate of Membership Inference Attacks (MIA). If a point has a high GNQ, an attacker is much more likely to correctly guess it was in the training set.
  • Explaining Hyperparameters: GNQ provides a theoretical lens to see why certain training choices affect privacy. For instance, it explains why smaller batch sizes or higher learning rates generally increase privacy risk (they increase the "uniqueness" of individual updates).

Key Results

The results show a massive "utility win" over traditional DP-SGD:

  • CIFAR-100: To reach a high level of privacy (MIA AUC of ~0.51), DP-SGD dropped model accuracy to a dismal 6.83%. In contrast, removing the top 20% of high-GNQ points achieved the same privacy level while maintaining 34.92% accuracy.
  • Tiny ImageNet: Similar story. DP-SGD accuracy was 2.15% for high privacy, while the GNQ-based defense held at 25.84%.
  • Model Inversion: On the Faces dataset, the authors showed that an attacker could clearly reconstruct the face of a high-GNQ individual. After removing just that one high-risk image and retraining, the reconstruction became a blurry, unrecognizable mess, while overall model accuracy barely budged.

Limitations and Future Work

While promising, the paper acknowledges a few hurdles:

  • Computational Cost: Calculating the full matrix $S$ and its inverse is impossible for giant models (like a 500B parameter LLM). The authors used a "diagonal approximation" (ignoring off-diagonal correlations) to make it feasible, but more efficient approximations are needed.
  • Assumptions: The mathematical proof relies on the assumption that gradients roughly follow a Gaussian distribution and that no single point's gradient is so unique that it exists in a subspace entirely its own.
  • Defense Sophistication: The current defense is binary (keep or drop). Future work could explore "targeted noise"—adding noise only to the specific gradient dimensions that make a point unique, rather than censoring the data point entirely.

Summary updated

Abstract

Disclosing private information via publication of a machine learning model is often a concern. Intuitively, publishing a learned model should be less risky than publishing a data set. But how much risk is there? In this paper, we present a principled disclosure metric called \emph{gradient uniqueness} that is derived from an upper bound on the amount of information disclosure from publishing a learned model. Gradient uniqueness provides an intuitive way to perform privacy auditing. The mathematical derivation of gradient uniqueness is general, and does not make any assumption on the model architecture, dataset type, or the strategy of an attacker. We examine a simple defense based on monitoring gradient uniqueness, and find that it achieves privacy comparable to classical methods such as DP-SGD, while being substantially better in terms of (utility) testing accuracy.
BibTeX
@misc{abdelghafar2026quantifying,
  author = {Sleem Abdelghafar and Maryam Aliakbarpour and Chris Jermaine},
  title = {{Quantifying Information Disclosure During Gradient Descent Using Gradient Uniqueness}},
  year = {2026},
  url = {https://openreview.net/forum?id=cYq0O9Z5iv},
  howpublished = {OpenReview},
  note = {Submitted to ICLR 2026},
}

Loading related content...

Figure