In federated learning, many clients train a shared model on their own data and send only parameter updates to a central server. To protect those clients, the server often receives the updates through secure aggregation, a cryptographic protocol that reveals only their sum. That same blindness makes backdoors easy to plant. A backdoor is a hidden rule, learned from poisoned data, that makes the model output an attacker-chosen label whenever a secret trigger appears in the input, while the model behaves normally otherwise. A server that sees only the sum cannot tell which client inserted it. We propose that the server stop looking at parameters it is not allowed to see and instead audit the behavior of models it is allowed to see. Each round, clients are randomly partitioned into small groups. Each group is securely aggregated separately. The server then probes each group's aggregate model with local surrogates, which are small linear models fitted to a model's outputs on perturbed copies of an input, in the style of LIME (local interpretable model-agnostic explanations). A backdoor-carrying update pushes the local decision surface toward one target class in the same way across many unrelated inputs. We expect that signature to be measurable even when the poison is split across clients or disguised in parameter space. Repeated random partitions then work as a group test that singles out persistently malicious clients.
The two-sentence version: the server cannot inspect individual updates, but it can inspect the model built from a small random group of them, and a local surrogate reveals whether that group model has started bending its decision surface toward a single class everywhere. Reshuffling groups every round lets the server accumulate suspicion on the specific clients who keep appearing in bent groups, without ever seeing any client's update alone.
Core Claim
We claim that a differential local-surrogate statistic, computed on subgroup aggregates, detects and attributes backdoor contributions under secure aggregation. The statistic compares a group aggregate against the previous global model at fixed unlabeled anchor inputs using shared perturbation seeds. We expect it to work in settings where parameter-space checks fail: distributed triggers, heterogeneous (non-IID) client data, and adaptive attackers who scramble which parameters they change. The server needs no labeled validation data and no knowledge of the trigger.
The idea is built from four pieces of prior work, each playing a different role. Model X-ray, from Detecting Backdoored Models via Decision Boundary, shows on centralized models that backdoors shrink clean decision regions and expand the target region in 2D boundary slices. This motivates our signal. We move from whole-model detection to per-round, per-group detection of a change in the decision surface. STRIP shows that backdoor behavior is input-agnostic under perturbation. This supplies the intuition that a backdoor's effect is consistent across anchors, although STRIP is a defense that screens incoming inputs at prediction time and does nothing during aggregation. Fooling LIME and SHAP is a constraint: models can detect off-distribution perturbations and behave innocuously on them. We therefore treat a perturbation-aware attacker as a first-class adversary, not an afterthought. The transparency-security results in On the Trade-Off Between Transparency and Security in Adversarial Machine Learning found that disclosing defense configuration gives attackers up to a 3.73x "matching advantage" in transfer attacks on CIFAR-10. That finding dictates that the audit's anchors, perturbation distribution and kernel stay secret and re-randomized. The explanation is a private instrument of the auditor, not a published artifact.
Relation to the Closest Work
Four lines of work sit closest to this proposal. xFedAlign has each client fit local surrogates and send differentially private top-k attributions, but only to correct benign explanation drift. Its explanations are self-reported, so a malicious client would simply report clean ones. In our design no client ever reports an explanation; the server computes it on models the client cannot tamper with after submission. INDEXGUARD vets clients under secure aggregation from the top-k parameter indices of their LoRA adapters. It works well in that setting, but it is tied to a shared adapter architecture, it inverts when attackers are the majority cluster, and it is exposed to attackers who scatter their changes across indices. Our statistic lives in input-output space, so parameter scattering does not hide it. BDPFL uses Grad-CAM heatmap distillation on the client side in personalized federated learning. XRand randomizes released feature rankings to stop explanation-guided trigger crafting. Both treat explanations as a white-box training tool or as an attack surface to be noised. We treat them as a hidden server-side probe. Finally, Accuracy is Not Enough shows federated attackers can distort Grad-CAM maps while keeping accuracy above 96%. That warns us that explanation-based audits can themselves be targeted, and it motivates our adaptive-attacker experiments.
Research Program
The protocol runs as follows. At round $t$ the server holds global model $\theta_t$. It samples a fresh random partition of the participating clients into groups of size $k$ and runs secure aggregation separately within each group, producing group aggregates $\theta_S$. Because the groups partition the clients and are reshuffled every round, the within-round group sums add up to the ordinary global update. No client's individual update is solvable from one round's aggregates, and updates change across rounds, which blocks naive cross-round differencing. We quantify the residual leakage empirically rather than assume it away.
The server keeps a secret pool of $M$ unlabeled anchor inputs. These can come from public out-of-domain data or a generator; labels are never needed. It also keeps a secret, per-round randomized perturbation distribution that mixes masking of patch or token groups, blending with other anchors, and patch shuffles. For each model, anchor $m$ and class $c$, it fits a kernel-weighted ridge surrogate with intercept $b^c_m$ and coefficients $w^c_m$ over feature groups, using identical perturbation seeds for $\theta_S$ and $\theta_t$. The group's backdoor score measures a consistent push toward one class from anchors whose current prediction is some other class:
$$D_S = \max_c \frac{1}{|A_{\neg c}|}\sum_{m \in A_{\neg c}} \frac{b^c_m(\theta_S) - b^c_m(\theta_t)}{\sigma^c_m} + \lambda \, \max_{c,j} \frac{1}{|A_{\neg c}|}\sum_{m \in A_{\neg c}} \big[w^c_{m,j}(\theta_S) - w^c_{m,j}(\theta_t)\big]_+ .$$
Here $A_{\neg c}$ is the set of anchors not currently predicted as $c$. $\sigma^c_m$ is the spread of that intercept change across all groups in the round, which normalizes away ordinary non-IID drift. $j$ indexes feature groups. $\lambda$ weights the second term, which captures a single feature group gaining influence toward one class everywhere, the footprint of a patch-like trigger. Benign heterogeneity should move surfaces in anchor-dependent, class-mixed directions. A backdoor should move them toward one class across anchors from many source classes.
Group scores are converted into per-client suspicion by accumulating each client's group scores across rounds. This is a noisy, adaptive group test. Clients whose running score exceeds a threshold calibrated on benign-only runs are excluded from aggregation. The analysis we will carry out is simple. With a fraction $p$ of malicious clients and group size $k$, a malicious client shares its group with benign ones, so its signal is diluted by roughly $1/k$. Suspicion therefore separates after a number of rounds that grows with $k$, which sets the tradeoff between privacy, since larger groups hide more, and time-to-detection. We will measure this curve rather than claim a closed form.
We will also build the strongest adaptive attacker we can. It knows the protocol but not the secret anchors or perturbation distribution. It trains its update with a penalty that keeps the aggregate's surrogate intercepts unchanged on a self-generated imitation of our perturbations, in the style of the scaffolding in Fooling LIME. It also tries low-and-slow poisoning at small per-round magnitude. The quantity that matters against it is the attack success rate reached before quarantine, not detection alone.
Baselines
The naive reference point is plain FedAvg under secure aggregation with no defense. It establishes how quickly each attack reaches high attack success rate in our setup. DBA reports near 100% attack success on MNIST, reached faster than centralized attacks. Classical robust aggregation, meaning coordinate median, Krum and trimmed mean as implemented in BackFed, is included even though secure aggregation formally forbids it. It shows what giving up privacy buys. BackFed reports these rules lower attack success at a large cost in clean accuracy, and that DeepSight costs 25.2 seconds per round versus 0.06 for FedAvg. We will report our audit's wall-clock cost on the same harness.
The strongest privacy-compatible competitor is INDEXGUARD, which reports an AUC of 1.0 or above 0.99 in malicious-client detection on SST-2 and AG News with LoRA adapters. We run it in its home setting, with identical clients, attacks, malicious fraction and rounds. Beating it there is not required. Matching it within noise while also working in full-model vision training, with heterogeneous client architectures sharing a common output space, and against index-scattering attackers would count as a win. The strongest method that uses clean data is TAG, reported to tolerate up to 40% malicious clients using a clean labeled validation set. We give TAG labeled versions of our anchors as an upper reference. Our goal is to come within a few points of its attack-success reduction without labels.
The decisive ablations isolate the surrogate. The first keeps the identical group-testing protocol but replaces $D_S$ with parameter-space statistics on group aggregates: update norm and cosine distance to the round mean. The second replaces $D_S$ with raw prediction drift on anchors with no surrogate fit. The third runs STRIP entropy on each group aggregate, and the fourth runs Model X-ray on each group aggregate. Every variant gets the same anchors, rounds and threshold calibration on benign runs. The surrogate earns its place only if it beats all four on time-to-detection at fixed benign false-rejection rate.
Why not just…?
Why not just run STRIP or Model X-ray on the group aggregates? This is the most serious threat, and the experiments above settle it directly. Both are strong on centralized, already-backdoored models: STRIP reports 0% false acceptance and 0% false rejection in most CIFAR-10 and GTSRB cases, and Model X-ray reports 100% detection on CIFAR-100. Our setting differs in that each group aggregate carries at most a diluted, single-round contribution, so the signal is a small change rather than an implanted backdoor. Our statistic is differential against $\theta_t$ with shared seeds, which should expose small shifts earlier. If X-ray or STRIP on group aggregates detects equally fast, we will report that the contribution is the group-audit protocol itself and that the detector is interchangeable.
Why not just apply parameter-space checks to the group aggregates? DBA shows that decomposed triggers evade median and similarity defenses because each piece resembles non-IID variance, and INDEXGUARD itself flags index-scattering adaptive attackers as a weakness. The norm and cosine ablation tests whether grouping alone rescues parameter-space checks. If it does, the surrogate is unnecessary.
Why not just keep a clean validation set, as TAG does? In the cross-silo medical and financial cases that motivate secure aggregation, the server often has no labeled reference data (Yi et al.). Per-client logit checks also need per-client models, which secure aggregation withholds. Our method needs only unlabeled, possibly out-of-domain anchors. The TAG-with-labels reference measures what labels would add.
Why not just measure output drift on anchors without fitting a surrogate? The surrogate's neighborhood fit should average out the sampling noise of single points and localize a consistent feature group, which is the second term of $D_S$. The raw-drift ablation will show whether this matters. If it does not, we will recommend the cheaper statistic.
Proposed Experiments
All vision experiments run on the BackFed harness with CIFAR-10, FEMNIST and Tiny-ImageNet, using Dirichlet non-IID splits. Attacks include pixel-pattern backdoors, DBA's distributed triggers, edge-case backdoors (Wang et al.), Neurotoxin, and our surrogate-aware adaptive attacker, at malicious fractions from 5% to 40% and group sizes $k$ of 5, 10 and 20. Language experiments reproduce INDEXGUARD's federated LoRA setting on SST-2 and AG News. Tabular experiments use LOAN following DBA, where local surrogates are most natural.
We report the following metrics. The first is attack success rate at the end of training and the peak reached before quarantine. The second is clean accuracy, and the third is the true- and false-exclusion rates for clients. We also report rounds-to-detection and audit compute per round as a multiple of FedAvg. Privacy leakage is measured by running membership inference and gradient inversion against group aggregates at each $k$ and against individual updates as the worst case.
The decisive checks are four. First, on DBA and edge-case attacks, the surrogate score must beat all four ablations on rounds-to-detection at matched false-exclusion rate. Second, against the adaptive attacker, the peak attack success before quarantine must stay well below FedAvg's. If scaffolding against a secret perturbation distribution proves cheap, we will publish that as the main negative result. Third, the benign-only false-exclusion rate must stay low under strong non-IID splits. Fourth, membership inference advantage on size-$k$ groups must sit close to the full-aggregate level rather than the individual-update level.
References
- : Detecting Backdoored Models via Decision Boundary
- STRIP: a defence against trojan attacks on deep neural networks
- Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods
- On the Trade-Off Between Transparency and Security in Adversarial Machine Learning
- Explainable Federated Learning via Global–Local Attribution Alignment
- INDEXGUARD: Index-only Backdoor Vetting for Secure Federated PEFT of Large Language Models
- BDPFL: Backdoor Defense for Personalized Federated Learning via Explainable Distillation
- XRand: Differentially Private Defense against Explanation-Guided Attacks
- Accuracy is Not Enough: Poisoning Interpretability in Federated Learning via Color Skew
- DBA: Distributed Backdoor Attacks against Federated Learning
- BackFed: A Standardized and Efficient Benchmark Framework for Backdoor Attacks in Federated Learning
- Trusted Aggregation (TAG): Backdoor Defense in Federated Learning
- Near-Optimal Resilient Aggregation Rules for Distributed Learning Using 1-Center and 1-Mean Clustering with Outliers
- Attack of the Tails: Yes, You Really Can Backdoor Federated Learning