EmInspector: Combating Backdoor Attacks in Federated Self-Supervised Learning Through Embedding Inspection

Yuwen Qian, Shuchi Wu, Kang Wei, Ming Ding, Di Xiao, Tao Xiang, Chuan Ma, Song GuoarXiv 2024 · DOI · OpenReview

TL;DR

This paper introduces "EmInspector," a novel defense mechanism against backdoor attacks in Federated Self-Supervised Learning (FSSL). EmInspector effectively detects malicious clients by analyzing the clustering behavior of their model embeddings, outperforming existing defenses.

What Problem Does This Paper Address?

Federated Self-Supervised Learning (FSSL) is a powerful paradigm for training models on decentralized, unlabeled data while preserving privacy. However, like its supervised counterpart (FSL), FSSL is vulnerable to backdoor attacks. In FSL, attackers inject triggers into data to associate them with specific labels. In FSSL, the attack is more subtle: attackers aim to manipulate local models to produce similar embeddings for triggered inputs, which then influence the global model's representation. Existing defenses, often designed for FSL or assuming outlier behavior from malicious models, are insufficient for FSSL because backdoored FSSL models don't necessarily produce outlier embeddings; instead, they tend to cluster together. This paper bridges this gap by investigating FSSL backdoor attacks and proposing a defense that leverages this unique clustering behavior.

Methods

The core of the paper is the EmInspector defense mechanism. It operates on the observation that backdoored local models in FSSL, when presented with a specific "inspection image," tend to generate embeddings that cluster closely together in the embedding space. Benign models, on the other hand, produce more diverse and scattered embeddings.

EmInspector works as follows:

  1. Inspection Dataset: A small, unlabeled dataset of "inspection images" is used. These images don't need specific labels or distributions.
  2. Embedding Similarity Calculation: For each inspection image, the server calculates the similarity of embeddings generated by each uploaded local model. This is done by computing the accumulated similarity of a model's output embedding with all other local models' embeddings.
  3. Decision Boundary: A decision boundary is established based on the average and median similarity values across all local models for a given inspection image.
  4. Malicious Score Update: Each local model's "malicious score" is updated based on whether its similarity to others exceeds the decision boundary.
  5. Malicious Client Identification: After processing all inspection images, clients with a positive malicious score are identified as malicious and their models are removed before global aggregation.

The paper also details the attack methodology, including:

  • Injecting Triggers: Malicious clients aim to generate similar feature vectors for triggered data and target class reference data, while also maintaining primary task accuracy.
  • Collusion Methods: Two methods are explored: "single pattern attack" (all malicious clients use the same trigger) and "coordinated pattern attack" (each client uses a unique trigger, assembled globally).
  • Attack Timing: Attacks can be launched at any round, but are more effective when the global model is close to convergence.

Datasets

The paper uses the following standard datasets:

  • CIFAR10: A common dataset for image classification.
  • STL10: Another image dataset, also used for self-supervised learning.
  • CIFAR100: A more challenging dataset with 100 classes.
  • GTSRB: Used for evaluating performance when pre-training and downstream datasets are distinct.

No new benchmarks were introduced; the focus was on evaluating existing FSSL systems and defenses.

Key Insights and Ideas

  • FSSL is Vulnerable: The paper convincingly demonstrates that FSSL systems are susceptible to sophisticated backdoor attacks, even when malicious clients are a minority.
  • Backdoored Embeddings Cluster, Not Outlier: A crucial insight is that backdoored FSSL models don't necessarily produce outlier embeddings. Instead, they tend to create clustered embeddings that are similar to each other, making traditional outlier-detection defenses ineffective.
  • EmInspector's Novelty: EmInspector leverages this clustering behavior, offering a new paradigm for detecting malicious clients in FSSL by focusing on similarity rather than dissimilarity.
  • Minimal Data Requirement: EmInspector is surprisingly effective with a very small set of inspection images (e.g., ten images for CIFAR100), making it practical.
  • Attack Stealth: Backdoor attacks can be launched without significantly degrading the primary task accuracy, making them hard to detect.

Key Results

  • High Attack Success Rate (ASR): Backdoor attacks achieved ASRs of over 95% with as few as 20% malicious clients on CIFAR10 and STL10.
  • EmInspector Effectiveness: EmInspector significantly reduced ASRs to very low levels (e.g., <1% on CIFAR100) while maintaining high primary task accuracy (ACC), outperforming existing defenses by a large margin. For instance, EmInspector reduced ASR from over 90% to around 12% on CIFAR10.
  • Robustness: EmInspector demonstrated effectiveness across various settings, including different datasets, i.i.d. and non-i.i.d. data distributions, network architectures, and attack degrees.
  • Minimal Inspection Data: As few as 10 inspection images were sufficient for EmInspector to effectively mitigate attacks on CIFAR10 and CIFAR100.
  • Attack Timing Impact: Attacking later in the training process led to faster ASR convergence, but attacking early did not negatively impact primary task accuracy.

Limitations and Future Work

  • Adaptive Attackers: While EmInspector is robust to several adaptive attacks (e.g., using different reference images or clean models), the paper acknowledges that more sophisticated adaptive attacks might exist.
  • Prior Knowledge of Malicious Client Proportion: The paper suggests that if the defender has prior knowledge of the proportion of malicious clients, the decision threshold can be adjusted to further reduce false positives. This implies that without such knowledge, there's a trade-off between true positive rate (TPR) and false positive rate (FPR).
  • Computational Overhead: While EmInspector is described as computationally efficient, the exact overhead compared to other defenses is not detailed.
  • Future Work: The authors implicitly suggest exploring more advanced adaptive attack strategies and potentially incorporating prior knowledge about the system to further refine defenses.

Summary updated

Abstract

Federated self-supervised learning (FSSL) has recently emerged as a promising paradigm that enables the exploitation of clients' vast amounts of unlabeled data while preserving data privacy. While FSSL offers advantages, its susceptibility to backdoor attacks, a concern identified in traditional federated supervised learning (FSL), has not been investigated. To fill the research gap, we undertake a comprehensive investigation into a backdoor attack paradigm, where unscrupulous clients conspire to manipulate the global model, revealing the vulnerability of FSSL to such attacks. In FSL, backdoor attacks typically build a direct association between the backdoor trigger and the target label. In contrast, in FSSL, backdoor attacks aim to alter the global model's representation for images containing the attacker's specified trigger pattern in favor of the attacker's intended target class, which is less straightforward. In this sense, we demonstrate that existing defenses are insufficient to mitigate the investigated backdoor attacks in FSSL, thus finding an effective defense mechanism is urgent. To tackle this issue, we dive into the fundamental mechanism of backdoor attacks on FSSL, proposing the Embedding Inspector (EmInspector) that detects malicious clients by inspecting the embedding space of local models. In particular, EmInspector assesses the similarity of embeddings from different local models using a small set of inspection images (e.g., ten images of CIFAR100) without specific requirements on sample distribution or labels. We discover that embeddings from backdoored models tend to cluster together in the embedding space for a given inspection image. Evaluation results show that EmInspector can effectively mitigate backdoor attacks on FSSL across various adversary settings. Our code is avaliable at https://github.com/ShuchiWu/EmInspector.
BibTeX
@misc{qian2024eminspector,
  author = {Yuwen Qian and Shuchi Wu and Kang Wei and Ming Ding and Di Xiao and Tao Xiang and Chuan Ma and Song Guo},
  title = {{EmInspector: Combating Backdoor Attacks in Federated Self-Supervised Learning Through Embedding Inspection}},
  year = {2024},
  eprint = {2405.13080},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2405.13080},
}
Figure