Federated learning promises privacy by keeping raw data on local devices and sharing only model updates or gradients. However, a major line of research reveals that this setup contains a fundamental privacy leak: the gradients of a neural network's final classification layer act as a highly accurate proxy for the underlying label distribution of the training data.
Whether a coordinator wants to optimize distributed training or a malicious actor wants to steal private user profiles, the mathematical structure of standard loss functions makes distribution inference surprisingly straightforward.
The Core Mechanism: Why Gradients Leak Class Proportions
To understand why this leakage occurs, we must look at the final fully connected layer of a classification network, right before the softmax activation. When training a model on a multi-class classification task, the standard choice of loss function is cross-entropy.
For a single training sample, the gradient of the cross-entropy loss with respect to the bias term $b_c$ of class $c$ is given by the elegant relationship:
$$\frac{\partial L}{\partial b_c} = p_c - y_c$$
where $p_c$ is the network's predicted probability for class $c$, and $y_c$ is the ground-truth indicator (equal to $1$ if the sample belongs to class $c$, and $0$ otherwise).
As Yige Liu et al. note in Label Leakage in Vertical Federated Learning: A Survey, this derivative introduces a stark sign asymmetry. If class $c$ is the true label, $y_c = 1$, making the gradient $p_c - 1$, which is always negative. For every other class, $y_c = 0$, making the gradient $p_c$, which is always positive.
When a client updates their model over a local batch of data, these gradients accumulate. If a client's dataset is heavily skewed toward a specific class, the true label indicator $y_c = 1$ triggers repeatedly for that class, driving its corresponding bias and weight gradients strongly negative. Conversely, classes that are absent or rare will only accumulate positive gradients ($p_c$).
Consequently, the magnitude and direction of the final layer updates directly mirror the class frequencies. As shown by Huancheng Chen et al. in Accelerating Non-IID Federated Learning via Heterogeneity-Guided Client Sampling and Heterogeneity-Guided Client Sampling: Towards Fast and Efficient Non-IID Federated Learning, the final layer's bias vector updates provide a compact, low-dimensional footprint of a client's local label distribution, bypassing the need to inspect high-dimensional weight matrices or raw data.
Strategies for Reconstructing the Distribution
Researchers have developed several distinct strategies to translate these raw gradient shifts into precise class percentages, falling into three main paradigms:
1. Linear Decomposition and Optimization
If the model's output logits are known or can be tightly bounded, the relationship between gradients and class proportions can be framed as a system of linear equations. In Decaf: Data Distribution Decompose Attack Against Federated Learning, Zhiyang Dai et al. demonstrate that an observer can treat the extracted gradient changes from the last fully connected layer as a linear combination of base gradients. By solving a linear regression problem, they can extract the exact scalar factors representing class proportions.
On a global scale, Tian Liu et al. show in Facilitating Early-Stage Backdoor Attacks in Federated Learning With Whole Population Distribution Inference that an adversary can apply evolutionary optimization algorithms to observed global updates to passively reconstruct the aggregate data distribution of the entire decentralized population.
2. Auxiliary Data Probing
Another approach relies on passing a balanced, public auxiliary dataset through the client's updated model to calibrate the gradient responses. In Addressing Class Imbalance in Federated Learning, Lixu Wang et al. show that a central server can accurately infer global distributions by comparing aggregated client gradients against the baseline gradients produced by a small auxiliary set.
Similarly, Miao Yang et al. leverage this intuition in Federated learning with class imbalance reduction, exploiting the mathematical relationship between the expectation of squared gradients and the number of training samples per class to map model updates back to exact class ratios.
3. Sensitivity and Response Analysis
A model inherently becomes less sensitive to data types it has already processed in abundance. In PPA: Preference Profiling Attack Against Federated Learning, Chunyi Zhou et al. show that by probing a local model update with specific class samples and measuring the resulting weight shift, an observer can reverse-engineer whether a class was a majority or minority component of the private dataset.
Alternatively, Qinbin Li et al. demonstrate in Exploiting Label Skews in Federated Learning with Model Concatenation that a server can send random noise to local models and analyze the resulting output probabilities; because the models are inherently biased toward their majority classes, the aggregated response patterns serve as an effective proxy for the underlying label distribution.
Two Sides of the Coin: Optimization vs. Exploitation
The ability to read data distributions from gradients is a double-edged sword. It can be harnessed constructively to fix systemic issues in federated networks, or weaponized maliciously to breach user privacy.
┌─────────────────────────────────────────┐
│ Final Layer Gradient Updates │
└────────────────────┬────────────────────┘
│
Is the distribution knowledge used for?
│
┌────────────────────┴────────────────────┐
▼ ▼
[ SYSTEM OPTIMIZATION ] [ PRIVACY EXPLOITATION ]
- Client Sampling (HiCS-FL) - Label Leakage Attacks (LIA-SA)
- Target Distribution Matching - User Preference Profiling (PPA)
- Long-Tail Re-balancing (CReFF, GBME) - Data Decomposition (Decaf)
Harnessing Gradients for System Optimization
Federated learning often struggles with non-IID (heterogeneous) data and long-tailed class distributions, which cause severe objective drift and slow down convergence. By treating final-layer gradients as a privacy-preserving proxy for data distribution, coordinators can intelligently re-balance the system without demanding raw metadata:
- Targeted Client Selection: In On-Demand Federated Learning for Arbitrary Target Class Distributions, Isu Jeong et al. analyze final softmax layer gradients to selectively combine specific clients, engineering an arbitrary, user-defined target class distribution on demand. Similarly, Zibo Wang et al. use weight changes from local batch gradient descent in Federated Analytics Informed Distributed Industrial IoT Learning With Non-IID Data to quantify data skewness and guide participant selection in industrial IoT settings.
- Mitigating Long-Tailed Distributions: To combat global class imbalances, Yaopei Zeng et al. introduce Global Balanced Experts for Federated Long-Tailed Learning, which aggregates final-layer gradient norms to estimate global class frequencies and train an ensemble of specialized experts. In a similar vein, Xinyi Shang et al. present Federated Learning on Heterogeneous and Long-Tailed Data via Classifier Re-Training with Federated Features, where clients upload average gradients per class, allowing the server to synthesize balanced "federated features" to re-train the global classifier.
Exploiting Gradients for Privacy Attacks
Conversely, the exact same mathematical relationships can be exploited to bypass privacy guardrails. Even standard cryptographic defenses like Secure Aggregation (SA)—which sums client gradients so the server only sees the collective update—are vulnerable.
In Breaking Secure Aggregation: Label Leakage from Aggregated Gradients in Federated Learning, Zhibo Wang et al. demonstrate that a malicious server can distribute "fishing models" with subtly modified batch normalization layers. This modification allows the server to disaggregate the summed gradients and isolate individual bias updates, recovering the private training labels of individual clients with perfect accuracy.
Key Foundational Work
Addressing Class Imbalance in Federated Learning (2021) — contributed 2 concept elements to this direction.
Federated learning with class imbalance reduction (2020) — contributed 2 concept elements to this direction.