Self-Weighting Inside the Clipping Ball: Free Adaptive Aggregation for Client-Level Private Federated Learning

Federated learning trains one model across many data holders, such as hospitals, without moving their data. Client-level differential privacy (DP) adds a formal guarantee on top: the released model reveals almost nothing about whether any single hospital took part. The standard way to get that guarantee is to clip each client's model update to a fixed length $C$ and then add Gaussian noise to the sum. The best non-private methods for heterogeneous federations do something else. They reweight clients: they shrink updates from clients with corrupted or low-quality data and protect clients who gain most from collaborating. These weights are computed from each client's private data. Released naively, they leak information, and the quantities behind them, such as loss gaps and sharpness, have no a priori bound. That last property is exactly what breaks textbook DP calibration. As a result, every private method in the literature either aggregates uniformly or drops update magnitude entirely.

This note proposes a simple remedy. Any client-local computation that keeps the client's contribution inside the length-$C$ ball costs no extra privacy. Adaptive weighting can therefore be moved out of the server and into the client's own clipping step: each client shrinks its own update by a weight in $[0,1]$ computed from its own data. The only remaining privacy expense is a cheap noisy count used to calibrate the weights across clients. In this view weighting and clipping become the same knob. The price of reweighting is also explicit: every unit of shrinkage removes signal while the noise stays fixed. We will build this method, derive when shrinking a client pays for itself, and test it on the medical cross-silo benchmarks where collaboration helps small sites most.

Core Claim

Under client-level DP with add/remove neighbors, the Gaussian mechanism protects any per-client vector of norm at most $C$, however that vector was produced from the client's data and the public global model. Client $k$ therefore sends

$$v_k = w_k \, C \, \frac{\Delta_k}{\|\Delta_k\|}, \qquad w_k \in [0,1],$$

where $\Delta_k$ is the local model change and $w_k$ is a bounded score computed locally. The server releases $\sum_k v_k + \mathcal{N}(0, \sigma^2 C^2 I)$, where $\sigma$ is the noise multiplier fixed by the privacy budget. It divides by a noisy total weight $\hat W \approx \sum_k w_k$, released at small extra cost.

The resulting average carries noise of standard deviation roughly

$$\frac{\sigma C}{\sum_k w_k}.$$

Two things follow. First, shrinking a client is free in privacy but not free in accuracy, because it inflates the effective noise. Second, inflating a client above $C$ is never cheap, because the noise must cover the largest possible contribution. Private adaptive weighting is therefore asymmetric: it can only suppress clients. A client should be suppressed only when the harm its update does to others exceeds the noise amplification caused by removing its mass. We expect this rule to recover most of the fairness and robustness gains of non-private sharpness-based and loss-gap-based weighting at the same $(\epsilon,\delta)$ as uniform private averaging. We also predict a measurable crossover: as $\epsilon$ shrinks, the optimal weights collapse toward uniform.

The construction directly addresses a tension in the literature. DP-NormFedAvg normalizes every update to length $C$, which keeps the signal-to-noise ratio constant for objectives with unbounded gradients but averages all clients uniformly. EAGLE and FedISM both need per-client magnitude information to work. EAGLE reweights by the gap between each client's loss and its standalone optimum; FedISM reweights by local loss sharpness. Our update keeps normalization's direction and constant scale, and reinjects exactly one degree of freedom per client, a privacy-safe scalar, inside the same sensitivity bound.

Research Program

The first piece is the local score. Following FedISM, which showed that upweighting high-loss corrupted clients drives the model into sharp minima, each client computes a sharpness estimate $\rho_k$ of the received global model on its own data. It also computes a cheap collaborative-benefit signal $g_k$: the held-out loss of the global model minus that of its own locally fine-tuned copy. This is a running proxy for EAGLE's loss gap that avoids EAGLE's costly standalone pre-training. A negative $g_k$ means the client already benefits from the federation; the file reports such negative gaps empirically.

Both signals are unbounded, especially under heavy-tailed losses. We never release them directly. Instead, $w_k$ is a sigmoid of $(\rho_k - \tau)/T$ and similarly for $g_k$, where $T$ is a fixed temperature. Only the threshold $\tau$ needs cross-client information, and it is set from a private quantile computed by noisy bisection over counting queries ("how many clients have $\rho_k < t$?"). A count changes by at most one when one client changes, whatever the support of $\rho$. This sidesteps the unbounded-sensitivity problem in the same spirit that Private Statistical Estimation via Truncation and the smooth-sensitivity estimators (Average-Case Averages) avoid worst-case bounds, though with a much simpler mechanism.

With very few clients, as in six prostate sites, the noisy counts are useless. That is where we combine the method with the sub-client splitting of Adaptive Intermediary, which turns each hospital into several sub-clients. The combination fits well: splitting raises heterogeneity across sub-clients, which is precisely what geometry-aware weighting is meant to tame.

The second piece is analysis. Understanding Clipping for Federated Learning decomposes DP-FedAvg error into optimization error, noise, and intra- and inter-client clipping bias. We extend that decomposition to weighted in-ball contributions. The weights move the fixed point toward a reweighted objective, which is intended. The noise term scales as $\sigma C / \sum_k w_k$. We will derive the suppression condition, under which removing mass from client $k$ helps when its inter-client drift term exceeds the marginal noise increase. We then check whether the learned weights track that condition. We will not claim non-convex guarantees.

The third piece measures collaborative benefit per site, as the global model's gain over that site's isolated model. FedMAP and the dental and mammography studies found that small sites benefit disproportionately from collaboration. DP noise and uniform clipping may erase exactly those gains, and the question is whether self-weighting restores them.

Baselines

The naive reference is DP-FedAvg with a fixed difference-clipping threshold at half the average update norm, as in Understanding Clipping, with sample-size aggregation. Alongside it we run local-only training for each site, so that collaborative benefit can be measured at all.

The strongest private competitors depend on the regime. On generic image benchmarks they are DP-NormFedAvg, which reports 59.36% versus 56.53% for clipped DP-FedAvg on CIFAR-100 at $\epsilon=5$, and DP-FedSAM-topk from Towards the Flatter Landscape, which reports about 56.36% versus about 49.06% on CIFAR-10 with Dirichlet 0.3 at $\epsilon=10$. In cross-silo medical imaging the strongest is Adaptive Intermediary, which reports 72.95% versus 62.01% for DP-FedAvg on RSNA-ICH at noise multiplier 1.5, and 58.14% versus 13.59% Dice on prostate MRI at multiplier 0.7. These are the strongest because each addresses one failure mode we target: vanishing signal-to-noise ratio, sharpness under noise, and the small-$N$ noise collapse.

Every method uses the same splits, architecture, rounds, local epochs, and privacy accountant at matched $(\epsilon,\delta)$, including the budget we spend on quantile counts. Every method gets the same hyperparameter-search budget over learning rate and $C$. Beating a baseline means higher worst-site accuracy or Dice and lower loss-gap variance at matched $\epsilon$, with mean accuracy no worse than one standard error across three seeds.

As non-private upper bounds we run FedISM and EAGLE. FedISM reports +10.85% on corrupted RSNA-ICH images over FedAvg; EAGLE reports loss-gap variance of 0.005 versus 0.081 for FedAvg on DirtyMNIST. We also run FedVal as an oracle with a server validation set; it is not deployable here, but it shows how much headroom weighting has.

Three ablations isolate the key ingredient. The first replaces the learned weights with random weights having the same distribution of $\sum_k w_k$, to rule out an effective smaller learning rate. The second uses one tuned uniform global step size, to separate weighting from step-size effects. The third uses the sharpness score alone, the gap score alone, and both together.

Why not just…?

Why not just privatize the scalar weights and run EAGLE or FedISM on the server? This is the real threat. Clients would release noisy sharpness or loss-gap scalars after clipping them, and the server would weight their clipped updates. The approach pays twice. The scalars are unbounded, so clipping them adds bias. Any server-side weight above one raises the sensitivity of the sum, so the noise must be calibrated to the maximum weight. Our design pays neither cost, but it gives up server-side rebalancing upward. We run this variant at matched $\epsilon$, tuning its scalar clip and its budget split between scalars and model updates. If it matches us, the in-ball idea reduces to a convenient implementation detail, and we will report it that way.

Why not just use DP-NormFedAvg? It removes unbounded sensitivity and is our default update geometry. However, it cannot tell a blurred-scanner site from a clean one. The non-private evidence from FedISM and EAGLE shows that uniform averaging leaves large disparities across clients under quality shift. Our method is DP-NormFedAvg in the special case $w_k \equiv 1$, so the comparison isolates exactly the value of weighting.

Why not just run local sharpness-aware training (DP-FedSAM) and aggregate uniformly? It flattens the landscape against noise. FedISM, however, found that local sharpness-aware training without sharpness-aware aggregation does not close gaps under quality shift. We test DP-FedSAM, and DP-FedSAM combined with self-weighting, to see whether the two effects add or overlap.

Why not just split clients into intermediaries? That shrinks the noise but increases heterogeneity across sub-clients, as the Adaptive Intermediary authors note. We treat it as complementary, and we test whether self-weighting on top of splitting beats splitting alone.

Proposed Experiments

The primary testbed is RSNA-ICH in FedISM's quality-shift setting, with simulated blur and sensor noise at a subset of sites, combined with Adaptive Intermediary splitting. Secondary testbeds are prostate MRI across six sites, DirtyMNIST across five clients with 0–100% ambiguous digits, FEMNIST with Dirichlet 0.1 label skew, and CIFAR-10/100 to match the published DP-FedSAM and DP-NormFedAvg settings. ISIC 2019 provides a second quality-shift check; no reported result was found on Lacuna for ISIC 2019 numbers. Metrics are mean and worst-site accuracy or Dice, accuracy on corrupted sites, per-site collaborative gain over local-only training, and variance of loss gaps, swept over $\epsilon \in \{1, 3, 5, 10\}$.

Three checks decide the claim. First, on RSNA-ICH at noise multiplier 1.5, self-weighting plus intermediaries must exceed Adaptive Intermediary's 72.95% while improving corrupted-site accuracy; the random-weight control must not. Second, the learned $\sum_k w_k$ should rise toward $N$ as $\epsilon$ falls, and the gain over DP-NormFedAvg should shrink accordingly. That would confirm the predicted noise-amplification crossover. If gains persist at tiny $\epsilon$, the weights are acting as a step-size effect, and the claim is wrong. Third, small sites' collaborative gain under DP should move closer to its non-private value than it does under uniform private averaging. That is the result that would show privacy need not cost the sites that collaboration was meant to serve.