Securing Federated Learning against Backdoor Threats with Foundation Model Integration

Xiaohuan Bi, Xi LiarXiv 2024 · DOI · arXiv

This paper, recently published by researchers from Renmin University of China and the University of Alabama at Birmingham, tackles a burgeoning security risk at the intersection of Federated Learning (FL) and Foundation Models (FMs).

TL;DR

The researchers identify a novel "top-down" backdoor attack where compromised Foundation Models on a server infect Federated Learning systems via poisoned synthetic data. They propose a unified, data-free defense that mitigates both classic and novel backdoor threats by dynamically optimizing upper bounds on internal model activations, effectively "clipping" malicious behavior without needing access to private client data.

What Problem Does This Paper Address?

Federated Learning (FL) is the go-to framework for training AI on decentralized data (like medical records or phone usage) without compromising privacy. Recently, developers have started integrating Foundation Models (FMs)—like GPT or Stable Diffusion—into FL to help generate synthetic data that assists in model aggregation.

However, this integration creates a massive security loophole. In a classic backdoor attack, a few malicious clients send "poisoned" updates to the server. Existing defenses catch these by looking for outliers. But in the novel FM-FL attack, the server’s FM itself is exploited to generate poisoned synthetic data. This poisoned data is used to fuse the global model, which is then sent to all clients. Because every client receives the same infected model, their updates look perfectly normal to the server. Traditional anomaly detection is rendered blind.

Methods

The authors propose a four-stage framework, adding a critical "Backdoor Mitigation" step to the standard FL cycle:

  1. Initialization: The server uses an FM to generate a synthetic dataset ($D_{syn}$) that mirrors the distribution of client data.
  2. Client Update & Global Fusion: Standard FL procedures where clients train locally and the server aggregates their parameters.
  3. Server Backdoor Mitigation (The Core Innovation): The authors observed that backdoor triggers—whether classic or FM-based—typically cause abnormally high activations in the model's hidden layers. To counter this, they introduce Activation Bounding.
  4. Logit Alignment: Instead of training on potentially mislabeled synthetic data, they optimize the upper bounds ($Z$) so the "bounded" model’s output matches the original model’s output on synthetic data. This preserves functionality.
  5. Constraint Optimization: They minimize the $l_2$-norm of these bounds using a Lagrangian function. This effectively "squeezes" the activations, finding the lowest possible ceiling that doesn't hurt clean-data accuracy but truncates the massive activation spikes caused by backdoor triggers.
  6. Hyper-parameter Adjustment: They use a threshold ($\Delta\pi$) to ensure that the defense doesn't drop the model's accuracy on synthetic data by more than a set amount (e.g., 10%).

Datasets

The researchers evaluated their defense using:

  • CIFAR-10 and CIFAR-100: Standard image classification benchmarks.
  • Synthetic Data Generation: They used GPT-4 to generate prompts for Dall-E, creating 10,000 synthetic images per dataset to facilitate the server-side defense.
  • Models: ResNet-18 was used as the backbone for the FL clients.

Key Insights and Ideas

  • The "Unified" Pattern: The "aha moment" here is the realization that despite having different origins (malicious clients vs. poisoned FMs), all backdoor attacks manifest as abnormal internal activations. This allows for a single defense mechanism to handle diverse threat vectors.
  • Data-Free & Privacy-Preserving: The defense happens entirely on the server using synthetic data. It never requires seeing the clients' raw, private data, making it fully compliant with FL's privacy goals.
  • Efficiency: Unlike "Trigger Inversion" defenses that are computationally expensive and slow, optimizing activation bounds is fast and requires fewer than five iterations per round, making it practical for continuous FL training.

Key Results

The defense was tested against several attack types, including BadNet (novel) and Blend/SIG (classic), across both homogeneous and heterogeneous FL settings.

  • Against Novel Attacks: In "Vanilla" FM-FL setups, the Attack Success Rate (ASR) often exceeded 90%. The proposed defense slashed the ASR to between 2% and 20% (depending on the dataset and IID settings) while keeping accuracy drops within the 10% limit.
  • Outperforming Baselines: Traditional defenses like Krum, ClipCluster, and SignGuard failed significantly against the novel FM-based attack, often leaving the ASR near 90%.
  • Robustness: The defense remained effective even in "non-IID" scenarios (where clients have very different data distributions), which is usually a major hurdle for FL security.

Limitations and Future Work

  • Accuracy Trade-off: The defense relies on a user-defined threshold ($\Delta\pi$). If set too strictly to kill the backdoor, it may degrade the model's performance on legitimate tasks.
  • Synthetic Data Quality: The effectiveness of the logit alignment depends on how well the FM-generated synthetic data mirrors the real-world client data. If the FM produces poor-quality data, the activation bounds might be sub-optimal.
  • Future Work: The authors suggest exploring how this defense scales to even larger models and more complex, non-image data types (like natural language or time-series data).

Summary updated

Abstract

Federated Learning (FL) enables decentralized model training while preserving privacy. Recently, the integration of Foundation Models (FMs) into FL has enhanced performance but introduced a novel backdoor attack mechanism. Attackers can exploit FM vulnerabilities to embed backdoors into synthetic data generated by FMs. During global model fusion, these backdoors are transferred to the global model through compromised synthetic data, subsequently infecting all client models. Existing FL backdoor defenses are ineffective against this novel attack due to its fundamentally different mechanism compared to classic ones. In this work, we propose a novel data-free defense strategy that addresses both classic and novel backdoor attacks in FL. The shared attack pattern lies in the abnormal activations within the hidden feature space during model aggregation. Hence, we propose to constrain internal activations to remain within reasonable ranges, effectively mitigating attacks while preserving model functionality. The activation constraints are optimized using synthetic data alongside FL training. Extensive experiments demonstrate its effectiveness against both novel and classic backdoor attacks, outperforming existing defenses.
BibTeX
@misc{bi2024securing,
  author = {Xiaohuan Bi and Xi Li},
  title = {{Securing Federated Learning against Backdoor Threats with Foundation Model Integration}},
  year = {2024},
  eprint = {2410.17573},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.17573},
}
Figure