Do Logic Constraints Leak or Shield Gradients in Federated Learning?

When several organizations jointly train a model without sharing raw data — the standard setup called federated learning — each client sends only the gradient of the loss on its private batch. A line of attacks has shown these gradients are far from opaque: an adversary who knows the model weights can recover the private training labels and even reconstruct the private inputs by treating the gradient as a system of equations in the unknown data. In parallel, many safety-critical domains (medical ontologies, autonomous driving) train networks with explicit logic rules — "if the image contains a stomach it contains a gastrointestinal system", "a red light and a green light cannot both be on" — embedded either as smooth penalties in the loss or as routing layers on top of the network. Nobody has asked what these rules do to gradient leakage. That question is not a curiosity: the rules themselves are usually public (published ontologies, a dataset's documented requirements), and every existing attack and defense assumes an unconstrained network.

Our core claim is that logic constraints cut both ways, and that which way they cut is predictable from where the constraint sits. Constraints in the loss (fuzzy-logic penalties, semantic loss) leave the network's gradient structure intact and may even hand the attacker a smaller search space, because public implication rules like "child implies parent" carve the label space into a smaller feasible region. Constraints in the architecture (max-routing layers over class hierarchies) break the clean algebraic structure that makes label recovery analytical. We propose to formalize this asymmetry, build the first constraint-aware gradient inversion attacks, and derive the first constraint-based, noise-free leakage defense — then map the utility–privacy tradeoff empirically on real constrained benchmarks.

Core Claim and What Prior Work Leaves Open

The sharpest existing attacks exploit one equation. For a standard linear output layer with cross-entropy, the gradient with respect to the weights of class $i$ is

$$\frac{\partial \mathcal{L}}{\partial W^{(L)}_{i,:}} = (p_i - y_i)\, a^{(L-1)},$$

where $p_i$ is the predicted probability, $y_i$ the true label, and $a^{(L-1)}$ the penultimate activation vector. Every row is a scalar multiple of the same vector, so the attacker can factor out $a^{(L-1)}$ and read labels off the coefficients. Sun et al.'s analytical attack Towards Eliminating Hard Label Constraints in Gradient Inversion Attacks reduces soft-label recovery to a single scalar and proves a striking negative result: the simplex constraint $\sum_i y_i = 1$ gives the attacker zero leverage, because every candidate label produced by their scalar reduction automatically satisfies it. Their attack recovers smoothed and mixed labels with over 99% accuracy on CIFAR-100 and 100% on ImageNet, and reconstructs images at 51.41 PSNR where baseline gradient-matching attacks hover between 8.01 and 10.22. RLU Recovering Labels from Local Updates in Federated Learning extends bias-vector label extraction to realistic multi-epoch, multi-optimizer federated training, keeping over 90% label accuracy on CIFAR-100 after 10 local epochs. Both assume mutually exclusive categorical labels and an unconstrained linear head — exactly the assumptions that multi-label, logic-constrained domains break.

The constrained-classification literature is equally silent in the other direction. C-HMCNN Coherent Hierarchical Multi-Label Classification Networks and CCN Multi-Label Classification Neural Networks with Hard Logical Constraints impose coherence architecturally: a parent's output must be at least the maximum of its children's, enforced by a max-routing layer on top of the base network. ROAD-R ROAD-R: the autonomous driving dataset with logical requirements formalizes 243 propositional constraints over 41 multi-label classes and shows that constrained training (CLCO-RCGRU) reaches 100% compliance while improving frame mean average precision from 30.78 to 31.81 over the unconstrained base model — a rare case where the constraint is free. The semantic loss A Semantic Loss Function for Deep Learning with Symbolic Knowledge penalizes the negative log-probability that the network's output distribution assigns to satisfying a boolean formula, doubling valid-path rates on structured prediction (6.99% to 69.89% on 4x4 shortest paths). None of these works considers a distributed setting or asks what a shared gradient reveals. Section 2 of our survey of the Lacuna index found no paper at this intersection.

Our theoretical wedge is a precise contrast with Sun et al.'s negative result. The sum-to-one simplex is uninformative because it is satisfied by every candidate in their attack family. Logical rules are different in kind: an implication $A \Rightarrow B$, relaxed under product logic to $y_B \ge y_A y_{child}$-style inequalities, restricts the label cube $[0,1]^K$ to a strictly smaller feasible polytope $\mathcal{F}_\alpha$. A public constraint set therefore acts as an informative prior over the private multi-label vector — the attacker's combinatorial search over $2^K$ configurations shrinks to the configurations consistent with $\alpha$, and for ROAD-R's 41 classes and 243 constraints that reduction could be enormous. This is a genuinely two-sided phenomenon, and neither side has been measured.

Research Program

Part 1 — Theory: the constrained gradient equation. We extend the analytical leakage equation to constrained architectures. With a max constraint module, the effective parent output is $M_A = \max\{h_A, \max_{B \in \text{children}(A)} h_B\}$, and the gradient reaching a base logit $h_j$ is

$$\frac{\partial \mathcal{L}}{\partial h_j} = \sum_{A} \frac{\partial \mathcal{L}}{\partial M_A}\, \mathbb{1}\!\left[j = \arg\max_{B \in \text{subtree}(A)} h_B\right],$$

a sparse sum routed only through argmax children. The rank-1 outer-product structure $\delta \cdot (a^{(L-1)})^T$ that Sun et al.'s and RLU's label extraction rely on no longer holds, because rows of the observed weight gradient mix contributions from several output units whose routing depends on the private activations. We will prove two lemmas: first, for loss-level constraints (t-norm penalties, semantic loss), the transmitted gradient is $\nabla_W \mathcal{L}_{\text{data}} + \lambda \nabla_W \mathcal{L}_{\text{semantic}}$, and since the attacker knows the weights and the public constraint set, the second term is a deterministic, computable function of the same unknowns — it changes the objective but adds no hiding power against an optimization attacker. Second, for architectural routing, the observed gradient is piecewise-linear in the data with breakpoints determined by private activations, and we characterize when the routing pattern itself becomes recoverable (small hierarchies, confident predictions) versus when it scrambles the factorization (deep, wide hierarchies, many tied activations). We also formalize the feasible-set lemma: the constraint polytope $\mathcal{F}_\alpha$ strictly reduces the attacker's label hypothesis space whenever $\alpha$ is non-trivial, quantifying the reduction for ROAD-R's 243 constraints.

Part 2 — Constraint-aware attacks. We build two attacks. The first is constraint-aware label recovery: multi-label label extraction is not a scalar-reduction problem (there is no sum-to-one simplex to exploit), but the attacker can enumerate or sample from $\mathcal{F}_\alpha$ and match against observed output-layer gradient signs and magnitudes, using RLU-style least squares over the feasible set rather than the full label cube. The second is constraint-aware image inversion: Geiping's cosine-similarity matching Inverting Gradients - How easy is it to break privacy in federated learning? is modified so dummy labels are constrained to $\mathcal{F}_\alpha$ during optimization, and so the routing pattern of the max layer is treated as a latent variable inferred jointly with the image. We hypothesize the constraint prior helps the attacker on shallow hierarchies but the routing scrambles recovery on deep ones — and we commit to reporting whichever direction the data shows.

Part 3 — Constraint placement as a defense. If architectural routing degrades inversion without noise, it becomes the first leakage defense that costs no utility: ROAD-R already shows constrained training improves accuracy while achieving 100% compliance. We design a defense study that varies constraint placement (none; loss-level t-norm penalty; output-level routing; both) at fixed task accuracy, and measures leakage. We also test the entanglement hypothesis of Concealing Sensitive Samples against Gradient Leakage in Federated Learning, which showed batch inversion succeeds because samples occupy near-orthogonal gradient subspaces: a shared constraint gradient term couples the samples' updates and may raise entanglement. We are honest that Part 1's first lemma predicts the coupling is removable by a knowing attacker, and the experiments will test that directly — an ablated attacker that subtracts the constraint term before matching.

Baselines

The do-nothing reference point is an unconstrained multi-label network with binary cross-entropy, trained in the same simulated federated loop as everything else, with no constraints anywhere. Its leakage anchors the scale of the problem: on such networks RLU reports 100% label recovery accuracy for untrained models after one local epoch and over 90% on CIFAR-100 after 10 local epochs Recovering Labels from Local Updates in Federated Learning, and Sun et al. report over 99% label accuracy on CIFAR-100 and 100% on ImageNet under smoothing and mixup Towards Eliminating Hard Label Constraints in Gradient Inversion Attacks. Any claimed defense must move these numbers materially, not marginally.

The strongest existing competitor is the analytical soft-label attack of Sun et al., which is the current state of the art for label recovery and for low-level vision architectures (51.41 PSNR on FCN-4 versus 8.01–10.22 for the $L_2$ and cosine-similarity gradient-matching attacks), combined with Geiping et al.'s cosine-similarity attack for image fidelity on ResNets (~14 dB PSNR on a trained ResNet-20, versus ~7 dB for the Euclidean baseline) and GradInversion for larger batches. We will run each of these unmodified against our constrained models — same federated simulation, same batch sizes, same attacker iteration budget and tuning — to measure how much constraint structure alone changes the game before we add constraint awareness. Then we run our constraint-aware versions with the same added compute or less, so any improvement is attributable to exploiting the constraints, not to extra search.

For the defense side, the baseline is DP noise on gradients, tuned to whatever variance is needed to push the strongest attack to our defense's leakage level; Sun et al. found their attack only fails when the noise variance reaches $10^{-1}$, at which the authors themselves state model utility is likely significantly degraded, so the comparison metric is leakage versus utility (f-mAP and compliance on ROAD-R, average precision on the hierarchical benchmarks) at matched protection. Finally, the key ablation isolates placement: train the unconstrained network and repair its outputs post hoc with a SAT solver (the Constrained Output, CO, setting from ROAD-R, where MaxHS repairs predictions in under 1 millisecond). CO achieves 100% compliance but transmits standard, unconstrained gradients, so any leakage difference between CO and in-training constrained models (CL/CLCO) is caused by the constraint being inside the gradient, not by compliant outputs.

Why not just…?

Why not just use differential privacy? DP gives a formal guarantee, and we will use it as the defense baseline rather than dismiss it. But the file's evidence shows the practical window is bad: Sun et al.'s attack survives low and moderate DP noise and only fails at noise variance $10^{-1}$, a level at which, per the same paper, utility is likely significantly degraded. In exactly the domains where logic constraints matter — ROAD-R's base model violates its requirements more than 90% of the time, and only constrained training reaches 100% compliance — a noise-degraded model risks losing precisely the coherence the constraints exist to enforce. Our experiments will settle this by plotting the full utility–leakage curve for DP versus constraint placement on the same models; if DP dominates our defense everywhere except compliance, we will report that plainly.

Why not just use label smoothing or mixup to diffuse the label signal? This was directly and decisively refuted: Sun et al. call smoothing and mixup "merely obstacles that can be overcome with rigorous mathematical modeling," recovering the diffused labels with over 99% accuracy on CIFAR-100 and 100% on ImageNet, and reconstructing images at 51.41 PSNR. Softening labels does not change the rank-1 gradient structure the attack exploits; our proposal attacks that structure itself via routing layers, which is a different mechanism.

Why not just post-process with a SAT solver at inference time? It is simpler, and it is also our central ablation, because it cleanly separates "compliant outputs" from "constrained gradients." ROAD-R shows post-hoc MaxHS repair gives 100% compliance in under a millisecond — but also that constrained training beats it on accuracy (f-mAP 31.81 for CLCO-RCGRU versus 30.78 base), and C-HMCNN shows post-processing leaves the underlying representation failure untouched, losing to in-training constraint layers on 17 of 20 hierarchical datasets. For privacy the point is sharper still: a post-processed model transmits exactly the gradients an unconstrained model transmits, so it should leak identically. If the experiments confirm identical leakage for CO and unconstrained models, that isolates our mechanism; if they do not, our theory is wrong and we will say so.

Why not just argue batch aggregation makes inversion impractical? A reviewer who believes this has not read the attack literature in the file: Geiping et al. reconstruct recognizable images from batches of 100 and invert ImageNet-scale images from a trained ResNet-152, explicitly showing deep networks add no protection; GradInversion recovers batches up to 48 on ResNet-50 with LPIPS 0.484 and about 28% of images identifiable at batch size 48; and RLU keeps over 90% label accuracy through 10 local epochs across VGG-16 and ResNet-50 under FedProx. We therefore run our simulations at batch sizes 8, 32, and 48 and multiple local epochs, in the regime where these attacks are already demonstrably effective.

Decisive Experiments

Three benchmark suites, all from the file, carry the empirical weight. First, ROAD-R: 41 multi-label classes, 243 propositional constraints, 22 driving videos; we simulate federated clients each holding video shards, training base, CL (fuzzy-logic constrained loss), CO (SAT post-processing), and CLCO variants. Metrics: per-class and exact-configuration label recovery accuracy, reconstruction PSNR and LPIPS, plus utility as f-mAP and compliance (the base model's f-mAP of 30.78 and CLCO-RCGRU's 31.81 with 100% compliance are the utility anchors we must not degrade). Second, the 20 hierarchical multi-label benchmarks of C-HMCNN and CCN — functional genomics, the IRMA medical imaging code, microalgae, text categorization — where C-HMCNN's average rank of 1.25 across all 20 datasets (first on 17) defines the utility bar, and where deep hierarchies let us test our prediction that routing scrambles the gradient factorization more as the tree deepens. Third, a controlled synthetic hierarchy with known depth, branching, and label statistics, plus the semantic-loss settings (MNIST semi-supervised, where semantic loss lifts accuracy from roughly 78% to 98.38% with 100 labels) to measure whether the arithmetic-circuit gradient term entangles or, as our lemma predicts, is subtractable.

The decisive checks are four. One: on the unconstrained and CO baselines, the strongest existing attacks should reproduce their published-scale numbers on our constrained-model code paths — sanity and calibration. Two: constraint-aware label recovery should beat constraint-blind label recovery on ROAD-R by a margin attributable to the feasible-set reduction $\mathcal{F}_\alpha$; if the constraint prior does not help the attacker at all, our feasibility lemma is misformulated and must be revised. Three: architectural routing (CLCO / C-HMCNN-style models) should cut analytical label recovery well below the >90% RLU regime while holding utility within one point of the published constrained numbers; if utility or leakage fails to move, the placement asymmetry is empirically empty and the proposal's defense claim dies. Four: at matched leakage reduction, constraint placement should retain 100% compliance and near-baseline f-mAP where DP noise sufficient to match (variance near $10^{-1}$ per Sun et al.) does not — the tradeoff that motivates the whole program. Whichever way the two-sided phenomenon resolves, the outcome is a publishable answer to a question every constrained federated deployment currently answers by guesswork.