BDPFL: Backdoor Defense for Personalized Federated Learning via Explainable Distillation

Guodong LongarXiv 2025 · DOI · arXiv

TL;DR

This paper introduces BDPFL, a novel framework for defending against backdoor attacks in Personalized Federated Learning (pFL). BDPFL uses layer-wise mutual distillation and explanation heatmaps to ensure local models are personalized and robust against malicious attacks, outperforming existing methods.

What Problem Does This Paper Address?

Federated Learning (FL) allows multiple clients to collaboratively train a global model without sharing their local data, preserving privacy. However, data heterogeneity across clients (Non-IID data) poses a significant challenge. Personalized Federated Learning (pFL) emerged to address this by allowing each client to have its own customized model. While pFL offers personalization, it inherits FL's vulnerability to backdoor attacks. Attackers can compromise a subset of clients and inject malicious triggers into the global model, causing it to misbehave on specific inputs while appearing normal otherwise. Existing backdoor defense strategies, primarily designed for general FL, often fall short in the pFL setting due to the inherent data heterogeneity, making it harder to detect malicious clients. This paper aims to fill this gap by proposing a robust defense mechanism specifically for pFL.

Methods

BDPFL introduces two key novelties:

  • Layer-wise Mutual Distillation: This technique enables each client to train two models: one for FL communication (w) and another for preserving personalized knowledge (w_p). During local training, clients receive the global model and use it to guide the training of their personalized model. Simultaneously, they train their communication model. Mutual distillation, using cross-entropy loss and KL divergence between global and local predictions, ensures knowledge transfer. Crucially, the "layer-wise" aspect emphasizes distilling knowledge from shallower layers more than deeper ones, as backdoor triggers are believed to propagate more in deeper layers. This is achieved by introducing a weight decay parameter that decays weights in deeper layers more rapidly.
  • Explanation Heatmap: To further enhance the distillation process and capture high-quality intermediate representations, BDPFL employs explanation heatmaps, specifically Grad-CAM. This method highlights important, class-specific features in intermediate layers. By distilling these heatmaps between the global and local models, BDPFL aims to align the behavior of potentially backdoored neurons with benign ones, effectively mitigating subtle backdoors. This distillation is applied from the global model to the local model, but not vice-versa, to bolster the defense.

The overall objective function for a client combines the mutual distillation loss and the explanation heatmap distillation loss.

Datasets

The paper evaluates BDPFL on three widely used image classification datasets:

  • MNIST
  • Fashion-MNIST
  • CIFAR-10

These datasets were split among 100 clients using Dirichlet allocation with a concentration coefficient of a = 0.5 to simulate data heterogeneity. For MNIST and Fashion-MNIST, a simple CNN model was used, while ResNet-18 was employed for CIFAR-10 to align with previous backdoor attack research.

Key Insights and Ideas

  • pFL's Unique Vulnerability: Data heterogeneity in pFL significantly complicates backdoor detection and defense, making generic FL defenses insufficient.
  • Layer-wise Importance: Backdoor triggers tend to manifest in deeper layers, suggesting that defenses should focus on preserving knowledge from shallower layers while still leveraging deeper insights.
  • Explainability for Defense: Using explanation heatmaps (Grad-CAM) provides a powerful way to align model behavior and identify deviations caused by backdoors, going beyond simple weight attenuation.
  • Mutual Distillation Synergy: Combining mutual distillation with explanation heatmap distillation creates a synergistic effect, improving both personalization and backdoor robustness.

Key Results

  • BDPFL significantly reduces the Attack Success Rate (ASR) for three common backdoor attacks (CSC, Neurotoxin, DBA) to below 5% across all tested datasets, while maintaining high Model Accuracy (ACC).
  • Compared to baseline methods (FG, NC, MK, RLR), BDPFL demonstrates superior performance, with baselines often showing significant degradation in ACC or failing to adequately reduce ASR in heterogeneous settings.
  • BDPFL shows robustness against more sophisticated pFL-specific attacks like BapFL and PFedBA.
  • The method is shown to be largely independent of model architecture and performs well across various data heterogeneity levels.
  • Ablation studies confirm the effectiveness of both layer-wise mutual distillation and the explanation heatmap component.

Limitations and Future Work

  • Hyperparameter Tuning: While BDPFL shows effectiveness, the paper notes that tuning hyperparameters like γ (for weight decay) requires careful consideration to balance defense strength and model performance.
  • Computational Overhead: Increased distillation epochs, while beneficial for defense, can lead to higher computational overhead. The paper suggests finding a reasonable balance.
  • Future Work: The authors propose extending defense mechanisms to the server side and exploring more robust aggregation algorithms. They also suggest deploying defense mechanisms on the server side and exploring robust aggregation algorithms.

Summary updated

Abstract

Federated learning is a distributed learning paradigm that facilitates the collaborative training of a global model across multiple clients while preserving the privacy of local datasets. To address inherent challenges related to data heterogeneity and satisfy personalized needs, a new direction within FL, known as personalized Federated Learning (pFL), has gradually evolved. Extensive attention has been directed toward developing novel frameworks and methods to enhance the performance of pFL. Regrettably, the aspect of security in pFL has been largely overlooked. Our objective is to fill this gap. Similar to FL, pFL is susceptible to backdoor attacks. However, existing backdoor defense strategies are primarily tailored to general FL frameworks, and pFL lacks robustness against backdoor attacks. We propose a novel, backdoor-robust pFL framework named BDPFL to address these challenges. First, BDPFL introduces layer-wise mutual distillation that enables clients to learn their personalized local models while mitigating potential backdoors. Then, BDPFL employs explanation heatmap to learn high-quality intermediate representations and enhance the effect of eliminating deeper and more entrenched backdoors. Moreover, we perform empirical evaluations of BDPFL's performance on three datasets and compare BDPFL with four backdoor defense methods. The experiments demonstrate that BDPFL outperforms baseline methods and is effective under various settings.
BibTeX
@misc{long2025bdpfl,
  author = {Guodong Long},
  title = {{BDPFL: Backdoor Defense for Personalized Federated Learning via Explainable Distillation}},
  year = {2025},
  eprint = {2503.06554},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2503.06554},
}
Figure