A Client-Side Isolation Monitor Against Active Gradient Inversion

In federated learning, many devices train one shared model without sending their raw data. Each device downloads the current model, trains it briefly on local data, and sends back only the change to the model's weights. Those weight changes can still reveal the data. The most damaging version is an active attack, in which a dishonest server alters the model weights before sending them out. It arranges things so that certain neurons fire for only one training example at a time. The weight change for such a neuron is then essentially a copy of that example. Reported reconstructions under these attacks are close to perfect: about 140 dB PSNR (peak signal-to-noise ratio, a pixel-fidelity score) on ImageNet in OASIS: Offsetting Active Reconstruction Attacks in Federated Learning. Clients currently have no way to tell, while training, whether the model they received has been rigged.

We propose a runtime monitor, borrowed in spirit from runtime verification (an independent observer that checks a live execution against a formal property and steps in when the property fails). It runs on the client between local training and upload. For every row of every linear layer it checks one property: the outgoing gradient row must be an even mixture of contributions from at least $k$ distinct training examples. Rows that fail are masked before the update leaves the device. The property is not a signature of any particular attack. It is the precondition of the analytic inversion that these attacks exploit. The monitor therefore checks the leakage mechanism itself, at almost no cost, and changes the update only when the mechanism is actually present.

Core Claim

Inverting Gradients - How easy is it to break privacy in federated learning? proves that the input to a fully connected layer can be read off exactly from the ratio of its weight gradient to its bias gradient, at any depth. For a batch, the weight gradient of linear layer $W$ is $\Delta^\top X$. Here $X$ holds the layer inputs (one row per example) and $\Delta$ holds the backpropagated error signals at that layer's outputs. Row $j$ of the gradient is therefore $\sum_i \delta_{ij} x_i$. It reveals an individual $x_i$ exactly when a single term dominates. Active attacks of the kind OASIS describes, "Robbing the Fed" and "Curious Abandon Honesty", engineer exactly this domination.

We define the effective number of contributors to row $j$ as

$$n_{\text{eff}}(j) = \frac{\left(\sum_i a_{ij}\right)^2}{\sum_i a_{ij}^2}, \qquad a_{ij} = |\delta_{ij}|\,\lVert x_i\rVert ,$$

where $a_{ij}$ is the size of example $i$'s contribution to row $j$. The quantity $n_{\text{eff}}$ equals $m$ when $m$ examples contribute equally and approaches 1 when one example dominates. The monitored property is $n_{\text{eff}}(j) \ge k$ for every row with a non-negligible gradient.

Both $\Delta$ and $X$ are already computed during ordinary backpropagation, so checking the property costs one pass over matrices the client already holds. When local training runs several steps, the monitor accumulates each example's contribution to each row across steps, keyed by example identity. The property then applies to the update that is actually released, not to any single step.

The claim we will test is threefold. Masking violating rows should defeat isolation-based active attacks as effectively as OASIS. It should rarely trigger on honest models. It should apply to any input type that passes through a linear layer, including tabular data, where image augmentation has no meaning.

Research Program

The work has three parts.

The first part builds and analyzes the monitor. We will prove a short lemma: if $n_{\text{eff}}(j) \ge k$, the weight-to-bias ratio for row $j$ returns a convex combination of inputs in which no single input carries weight above roughly $1/\sqrt{k}$. This lemma plays the role that a verified safety proof plays in ModelPlex: verified runtime validation of verified cyber-physical system models. ModelPlex compiles a proof about a physical model into a cheap arithmetic check that runs at every step. We compile the analytic inversion lemma into an equally cheap per-row inequality. The lemma covers only analytic inversion of linear layers. It says nothing about optimization-based inversion, and we will state that boundary plainly.

The second part designs what happens when the property fails. The architecture follows Safe Reinforcement Learning via Statistical Model Predictive Shielding, which places a shield between an unverified policy and the system it controls and substitutes a safe backup action when a check fails. Here the unverified component is the server's model, and the backup is a sanitized update. We will compare three fallbacks. The first zeroes the violating rows. The second zeroes the corresponding bias entries and rows together. The third withholds the whole round and reports the violation, which gives the client a signal that the server may be malicious.

The third part studies how often honest training trips the monitor. Ordinary ReLU networks can contain nearly dead neurons that fire for a single example in a small batch. The rate of such false alarms, and the accuracy lost to masking them, decides whether the method is usable at all.

The framing differs from three neighbors in the runtime verification literature. Monitoring Hyperproperties With Prefix Transducers motivates our view of leakage as a property relating many executions. Its monitors run over discrete symbolic traces, while our property is a continuous relation among per-example contributions within one update. Monitoring Robustness and Individual Fairness: A Runtime Approach with Clemont monitors deployed models, but only on forward-pass inputs and outputs at inference time. Our monitor watches training-time gradients.

Baselines

The naive reference point is undefended federated averaging under the Robbing the Fed and Curious Abandon Honesty attacks. This calibrates how bad leakage is. The reported figure to expect is about 140 dB PSNR on ImageNet with no defense, from OASIS.

The strongest existing method for this specific threat is OASIS. It applies rotations, shears, and flips during batch construction so that several examples co-activate any planted neuron. It is the right competitor because it targets the same attacks with the same mechanism, overlapping activations, but enforces that overlap statically and always. OASIS reports PSNR falling to about 15–20 dB on ImageNet, accuracy above 90%, and a maximum accuracy drop of 1.5% on CIFAR-100. We will rerun OASIS under the same attacks, model architectures, batch sizes, local-step counts, and client counts as our monitor, with the same training budget. Beating it means two things together. First, reconstruction PSNR and SSIM (a structural similarity score) at or below OASIS's level. Second, a smaller accuracy drop on CIFAR-100 and ImageNet than OASIS shows when we rerun it under identical settings.

On tabular data, OASIS's augmentations do not apply. There, beating it means blocking inversion where it offers nothing.

A second strong competitor is per-example differential privacy, which adds calibrated noise to each example's clipped gradient. We use Fed-CDP from Gradient-Leakage Resilient Federated Learning, which reports 97.48% MNIST accuracy against a 97.98% non-private baseline. We will tune its noise level so that it matches our monitor's reconstruction quality, then compare accuracy and client compute at that matched privacy level.

We will also include a weight-signature detector. It flags downloaded models showing the known planted structures, such as many nearly identical rows or sorted bias ladders, and refuses to train on them. This is the cheapest alternative a reviewer would propose.

Two ablations isolate the key ingredient. The first replaces $n_{\text{eff}}$ with a plain count of active ReLU units per row, testing whether contribution weighting matters. The second applies the check per local step rather than across the accumulated update.

Why not just…?

Why not just detect the planted weight patterns directly? A signature detector catches the attacks as published. An attacker who knows the detector can add small random perturbations to rows and biases so that the planted structure no longer matches the signature, while each neuron still fires for essentially one example. Our experiments include exactly this adaptive attacker. The monitor checks the isolation outcome rather than the weight structure, so it should remain effective while the signature detector fails. If the signature detector survives the adaptive attack too, the monitor's added value shrinks to its coverage of attacks nobody has yet named, and we will report that.

Why not just always run OASIS? OASIS is a real threat on images, where its accuracy cost is small: at most 1.5% on CIFAR-100. Its case is weaker elsewhere. It cannot be applied to tabular or embedding inputs, and it pays its cost in every round whether or not an attack is under way. If our benign-trigger experiments show that the monitor masks many rows in honest training, then OASIS on images is the better choice. We will say so and confine our claim to the tabular setting and to the attack-detection signal.

Why not just use per-example differential privacy or secure aggregation? Fed-CDP gives formal guarantees, but it pays a permanent noise cost and per-example clipping overhead. Its MNIST accuracy of 97.48% is close to baseline, but that figure comes from an easy dataset. Secure aggregation, which lets the server see only the sum of many clients' updates, fails when the server isolates a client or the cohort is small, as acknowledged in Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning. Secure aggregation therefore does not remove the need for a client-side check. The matched-privacy comparison against Fed-CDP will show whether a targeted mask beats uniform noise on accuracy at equal leakage.

Proposed Experiments

We will reimplement the Robbing the Fed and Curious Abandon Honesty attacks, following the descriptions in OASIS. The datasets are CIFAR-10, CIFAR-100, ImageNet, and TinyImageNet with the standard image models those papers use, plus Adult Income with a fully connected network. Batch sizes range from 8 to 128, local steps from 1 to 5, and $k$ from 2 to 16. The metrics are reconstruction PSNR, SSIM, and LPIPS (a perceptual similarity score); final test accuracy; the fraction of rows masked per round under honest and attacked models; and client wall-clock overhead.

The first decisive check is attack suppression. With $k \ge 4$, PSNR under both attacks should fall to OASIS's reported 15–20 dB range or below. The second decisive check is the honest-model false-alarm rate. Masking must remove under 1% of gradient rows in honest training, and accuracy must stay within OASIS's 1.5% CIFAR-100 drop; otherwise the method is not usable.

The third check is the adaptive attacker. This attacker optimizes planted weights to maximize what can be recovered subject to $n_{\text{eff}} \ge k$, then tries to separate the resulting $k$-example mixtures with a generative image prior of the kind used in Gradient Inversion with Generative Image Prior. If that prior separates the mixtures, $k$ must scale with what the prior can separate, and we will measure that relationship rather than claim a fixed threshold.

Finally, we will report how often the "withhold round" fallback flags malicious servers, compared with false flags on honest servers. A rate near zero for honest servers would make the monitor useful as a detector even where masking is not needed.