Backdoor-Safe Subspace Alignment for Federated Low-Rank Fine-Tuning

Large language models are increasingly fine-tuned in a federated way: many clients each train locally on private data and a server merges their updates. To keep this affordable, clients use LoRA — low-rank adapters, where each weight change is written as the product of two skinny matrices, $B A$ with inner dimension $r \ll d$, so only $2dr$ numbers travel instead of $d^2$. But LoRA has a hidden mathematical bug: any orthogonal rotation $R$ leaves the product unchanged, since $(B R)(R^{\top} A) = B A$. If two clients learn semantically identical updates in rotated coordinates, averaging their factors causes the updates to cancel. Two recent systems fix this by rotating every client's factors into a shared reference frame before averaging — FedRot-LoRA, which solves an Orthogonal Procrustes problem against the previous round's global model, and FLoRG, which averages rotation-invariant Gram matrices and pins the server-side re-decomposition FedRot-LoRA: Mitigating Rotational Misalignment in Federated LoRA, FLoRG: Federated Fine-tuning with Low-rank Gram Matrices and Procrustes Alignment. The fix is large: on MNLI with 50 clients, FedRot-LoRA reports 87.34% accuracy versus 76.80% for naive factor averaging.

Neither paper asks what this alignment machinery does when some clients are malicious. That question is urgent, because the backdoor literature has converged on exactly the property alignment exploits. A backdoor is a conditional behavior — normal on clean inputs, attacker-controlled when a trigger appears — and the most persistent federated attacks deliberately embed the Trojan update inside the "decision-critical, optimization-stable subspace" that aggregation naturally preserves: CTO-VPA sustains backdoor accuracy above 50% for over 250 rounds on CIFAR-10 in a single-shot attack, bypassing norm clipping, trimmed-mean, differential privacy, and FLAME Coupled Trigger Optimization and Vulnerable Parameter Alignment for Persistent Backdoor Attacks on Federated Learning. This proposal argues that subspace alignment and subspace-targeted backdoors are two ends of the same lever, and studies both ends: first whether Procrustes alignment is an unrecognized attack amplifier, then whether rotation-invariant aggregation can be turned into a defense that standard coordinate-wise filters cannot be.

Core Claim

The claim has two halves. First, negative: aligning client updates to a shared frame is not security-neutral. Procrustes alignment reduces aggregation error for every low-rank update, malicious ones included, so it should extend backdoor persistence; worse, because the alignment target is the previous global model, an attacker who corrupts early rounds can drag all subsequent benign updates toward a Trojan coordinate frame — a risk FedRot-LoRA itself flags but never tests. Second, positive: the same invariance principle that makes alignment possible supports a defense that coordinate-wise methods cannot imitate. Instead of averaging factor entries, the server can average subspaces — each client's update defines a column space and a row space, and a subspace has a canonical, rotation-free representation as an orthogonal projector. Fan et al. proved that averaging projectors is statistically optimal for distributed PCA Distributed estimation of principal eigenspaces, and Lim et al. showed projectors uniquely characterize eigenspaces where raw bases are ambiguous Sign and Basis Invariant Networks for Spectral Graph Representation Learning. Nobody has used this to filter Trojan subspaces in federated fine-tuning. Our defense aggregates projectors of only those clients whose subspaces form a robust consensus, and it does so at LoRA's communication cost, not the $O(d^2)$ cost that naively derailed projector methods.

Research Program

Part 1 — Is alignment an attack amplifier? We port two attacks into federated LoRA. The first is CTO-VPA-style: a minority of malicious clients jointly optimize a continuous trigger and a weight update confined to the aggregation-stable subspace (identified by the signal-to-noise influence scores that CTO-VPA defines), re-derived for the LoRA factor geometry. The second is the task-arithmetic attack from pipeline-parallel post-training, where an attacker adds a scaled delta vector encoding unsafe-trigger behavior; that attack reached 94% attack success on LLaMA-3.2 1B Instruct with the trigger word "SUDO" and retained 60% success even after a full downstream safety-alignment phase Backdoor Attacks on Decentralised Post-Training. We run both attacks under naive factor averaging (FedIT), FedRot-LoRA, and FLoRG, measuring backdoor accuracy and how many rounds the backdoor survives, alongside clean accuracy. We also test reference poisoning: malicious clients active only in early rounds, when the Procrustes anchor is noisy and most influenceable, with FedRot-LoRA's soft-rotation interpolation as the only existing mitigation.

Part 2 — Consensus projector aggregation as a defense. The defense replaces Procrustes alignment with subspace consensus. Per layer and per factor, each client sends its factor $\hat{U}_i \in \mathbb{R}^{r \times d}$ with orthonormalized rows; the server computes pairwise affinities $s_{ij} = \tfrac{1}{r}\|\hat{U}_i \hat{U}_j^{\top}\|_F^2$, which is the mean squared cosine of the principal angles between the two subspaces. Clients are ranked by total affinity; a trimmed inlier set is chosen around the subspace medoid, and the aggregated subspace comes from averaging the inliers' projectors. The crucial computational fact is that this never materializes a $d \times d$ matrix:

$$\text{top-}r \text{ eigenvectors of } \bar{P} = \tfrac{1}{|C|}\textstyle\sum_{i \in C} \hat{U}_i^{\top}\hat{U}_i \;\;=\;\; \text{top-}r \text{ right singular vectors of } \begin{bmatrix} \hat{U}_1 \\ \vdots \\ \hat{U}_{|C|} \end{bmatrix},$$

where $C$ is the inlier set, so the whole computation costs $O(|C| r d)$ — the same order as aggregating the factors directly, resolving the communication tension that Fan et al.'s one-shot analysis left open for high dimensions. Aggregated factors are then reconstructed by least-squares fitting each inlier's update onto the consensus subspace. As a secondary screen, per-layer spectral descriptors in the style of Z-PEFT (singular-value entropy, stable rank, energy concentration, computed cheaply from the small $r \times r$ QR product rather than the dense $BA$) flag rounds whose aggregated spectra look Trojan-like Z-PEFT: Zero-shot Backdoor Detection in Parameter-Efficient Fine-Tuning via Canonical Spectral Signatures. The design deliberately separates the two ingredients — invariance (projector form) and robustness (consensus trimming) — so ablations can attribute any gain.

Part 3 — Adaptive and heterogeneous attackers. Lockdown's authors concede that an adversary who infers the consensus mask can hide inside it Lockdown: Backdoor Defense for Federated Learning with Isolated Subspace Training; we test the subspace analogue, where the attacker confines the Trojan row/column space to the expected consensus. We also test rank-heterogeneous clients (AdaLoRA-style variable rank), the regime where Z-PEFT's static detection fails to generalize and FedRot-LoRA's fixed-rank Procrustes formulation breaks down. Our affinity extends naturally to subspaces of different dimension via partial principal angles, but whether consensus remains meaningful when ranks drift is an open empirical question we will answer rather than assume.

Baselines

The naive reference point is undefended factor averaging, FedIT as reported by FedRot-LoRA: 76.80% MNLI accuracy with 50 clients on RoBERTa-Large, where rotational cancellation already destroys utility before any adversary acts FedRot-LoRA: Mitigating Rotational Misalignment in Federated LoRA. We run it under identical poisoning so its backdoor dynamics anchor the comparison. The strongest existing competitor is FedRot-LoRA itself — 87.34% MNLI, and the system any deployment would actually use — stacked with the standard robust aggregators (norm clipping, trimmed-mean, FLAME) that CTO-VPA already evaluated, since that stack is the obvious "just add robustness" answer. It will be given the same clients, rounds, local compute, and tuning budget as our defense. Beating it means holding clean accuracy within one point of 87.34% while cutting single-shot backdoor persistence from CTO-VPA-scale (above 50% backdoor accuracy for over 250 rounds on CIFAR-10 under FedAvg with those same defenses, 71.43% peak) to under 50% within twenty rounds. Two further baselines bracket the design space: a Lockdown-style quorum rule adapted to LoRA coordinates — Lockdown reports up to 90% attack-success reduction with only a 2.7% clean-accuracy drop in vision federated learning, but no reported result found on Lacuna for Lockdown on LoRA or language models, so we reimplement its majority-vote coordinate pruning on adapter entries — and per-round Z-PEFT screening, where flagged clients are dropped before aggregation (its 0.943 leave-one-attack-out AUROC is on static PADBench adapters; no reported result found on Lacuna for Z-PEFT in a federated, per-round setting). Finally, the key ablation isolates our claim: projector-consensus averaging without outlier trimming (pure invariance) and trimmed coordinate averaging without invariance (pure robustness), each run with matched compute. If only the combination works, the interaction is the contribution; if either alone suffices, we report that honestly.

Why not just…?

Why not just use standard Byzantine-robust aggregation on top of alignment? This is the real threat, and we treat it as a first-class baseline rather than arguing around it. The evidence against it is specific: CTO-VPA bypassed norm clipping, trimmed-mean, differential privacy, and FLAME precisely because its update magnitudes sit inside the benign envelope along aggregation-stable directions CTO-VPA, and coordinate-wise filters are doubly confused in LoRA because they operate in each client's arbitrary rotated frame. But that evidence is from full-model vision training; if stacking FLAME on FedRot-LoRA turns out to suppress our federated-LoRA attacks as well as our defense does, the defense's practical value collapses and the paper's contribution narrows to the amplification finding of Part 1. Experiment 3 is designed to settle exactly this, head to head, same budget.

Why not just run a detector every round and drop bad clients? Z-PEFT's 0.943 zero-shot AUROC on PADBench makes this attractive, but it has three documented gaps: it uses only scalar spectra and ignores singular-vector orientation, which is precisely where our proposal operates; it fails to generalize to AdaLoRA's variable rank, which Part 3 shows is also an evasion route; and detection does not repair a Trojan already folded into the aggregated model, whereas projector consensus never admits the outlier subspace in the first place Z-PEFT. We still run it as a baseline because a per-round detector could plausibly catch grossly poisoned clients cheaply; if it matches our defense at lower cost, the honest conclusion is that detection suffices for this threat model.

Why not just prune or fine-mix after training? Pruning is the classic centralized remedy — dormant Trojan neurons removed without harming clean accuracy Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks — but Lockdown demonstrates the poison-coupling effect: in federated training, malicious and benign parameters entangle so that magnitude pruning trades attack success against clean accuracy, with competing defenses losing up to 43.6% accuracy in non-IID settings Lockdown. In a rank-$r$ adapter there are no dormant banks to prune in the first place, and PEFTGuard's fine-mixing mitigation linearly blends adapter weights with pretrained weights, degrading task accuracy while ignoring the subspace geometry PEFTGuard: Detecting Backdoor Attacks Against Parameter-Efficient Fine-Tuning. Post-hoc safety alignment is no better: the decentralised post-training backdoor retained 60% attack success through a dedicated safety phase Backdoor Attacks on Decentralised Post-Training. Prevention at aggregation time, not repair after, is the gap this proposal fills.

Proposed Experiments and Decisive Checks

All experiments use FedRot-LoRA's published settings so numbers are directly comparable: RoBERTa-Large on MNLI with 50 clients (reference accuracies 87.34% FedRot-LoRA, 82.44% RoLoRA, 76.80% FedIT), plus Llama-3-8B on HumanEval for scale (40.88% versus 28.77% pass@1 for naive FedLoRA) and SQuAD v1.1 in the FLoRG configuration. Poisoning ranges from a single-shot attack by one client to 10% colluding clients, with both the CTO-VPA-style and task-arithmetic trigger attacks. Metrics are clean accuracy, attack success rate, backdoor persistence in rounds, per-round communication, and false-positive impact on clean federations (a defense that trims benign clients as outliers fails even if it stops attacks). Decisive checks, stated in advance: Part 1 succeeds if alignment measurably extends backdoor persistence relative to naive averaging under matched attacks, and is falsified — and we will report it as falsified — if aligned and unaligned persistence curves are indistinguishable. Part 2 succeeds if consensus projector aggregation keeps clean accuracy within one point of FedRot-LoRA while cutting persistence by an order of magnitude in rounds, and fails if the adaptive consensus-hiding attacker of Part 3 erases the gap. Part 3's rank-heterogeneity result is informative either way: if variable rank breaks both our affinity measure and every existing fixed-rank method, that is itself the first characterization of an evasion route that, per the literature, no current defense covers.