BADFSS: Backdoor Attacks on Federated Self-Supervised Learning

Jiale Zhang, Chengcheng Zhu, Di Wu, Xiaobing Sun, Jianming Yong, Guodong Long

Abstract

Self-supervised learning (SSL) is capable of learning remarkable representations from centrally available data. Recent works further implement federated learning with SSL to learn from rapidly growing decentralized unlabeled images (e.g., from cameras and phones), often resulting from privacy constraints. Extensive attention has been paid to designing new frameworks or methods that achieve better performance for the SSL-based FL. However, such an effort has not yet taken the security of SSL-based FL into consideration. We aim to explore backdoor attacks in the context of SSL-based FL via an in-depth empirical study. In this paper, we propose a novel backdoor attack BADFSS against SSL-based FL. First, BADFSS learns a backdoored encoder via supervised contrastive learning on poison datasets constructed based on local datasets. Then, BADFSS employs attention alignment to enhance the backdoor effect and maintain the consistency between backdoored and global encoders. Moreover, we perform empirical evaluations of the proposed backdoor attacks on four datasets and compared BADFSS with three existing backdoor attacks that are transferred into federated self-supervised learning. The experiments demonstrate that BADFSS outperforms baseline methods and is effective under various settings.

1. Introduction

Self-Supervised Learning (SSL) has revolutionized AI by enabling models to learn from massive amounts of unlabeled data. When integrated with Federated Learning (FL), it creates Federated Self-Supervised Learning (FSSL), allowing collaborative training across decentralized devices without sharing private data. While FSSL enhances privacy, its security landscape remains largely unexplored. Most existing backdoor attacks target supervised learning, where labels are readily available for manipulation. In FSSL, the absence of labels and the central server's aggregation process typically mitigate malicious updates from individual participants. This paper introduces BADFSS, a sophisticated backdoor attack demonstrating that FSSL is highly vulnerable to persistent, hidden triggers.

2. Methodology: The BADFSS Attack

BADFSS is a two-stage attack designed to survive federated aggregation and remain stealthy throughout the training process.

The two-stage BADFSS attack: Backdoor Injection and Attention Alignment.

Stage 1: Backdoor Injection via Supervised Contrastive Learning

Since FSSL lacks global labels, the attacker creates "pseudo-labels" locally. A subset of local data is assigned a target class and augmented with a trigger (e.g., a specific pixel pattern). The attacker then employs Supervised Contrastive Learning to force the encoder to map any image containing the trigger into the same feature space as the target class.

Stage 2: Attention Alignment

To ensure the backdoor persists through multiple rounds of federated aggregation, BADFSS uses a novel "attention alignment" technique. The attacker aligns the local model's attention maps with those of the global model for clean images, ensuring stealth. For triggered images, the model is forced to maintain specific attention patterns that activate the backdoor. This makes the malicious features "sticky," allowing them to be successfully inherited by the global model during server-side aggregation.

3. Experimental Evaluation

The researchers evaluated BADFSS on four standard computer vision benchmarks: CIFAR-10, CIFAR-100, GTSRB (German Traffic Sign Recognition Benchmark), and Tiny-ImageNet. These datasets represent a range of complexity and real-world safety scenarios.

4. Key Results and Insights

BADFSS consistently outperforms baseline attacks adapted for the federated setting, such as BASSL or BadEncoder.

Attack performance (ASR and ACC) across varying ratios of malicious clients and poison samples.
  • High Success Rate: On CIFAR-10, BADFSS achieved an Attack Success Rate (ASR) exceeding 99%, while maintaining Main Task Accuracy (ACC) comparable to non-attacked models (~76%).
  • Robustness to Poisoning Ratios: The attack remains highly effective even with a very low "poison ratio," requiring as little as 1% of the attacker's local data to be manipulated.
  • Resilience to Decentralization: BADFSS is effective in "Non-IID" scenarios, where data distribution varies significantly across clients—the most common real-world FL setting.
  • Stealth and Persistence: By aligning attention maps, the attack minimizes the drop in clean accuracy, making it difficult for standard performance-based monitoring to detect. Furthermore, existing defenses like Neural Cleanse and DECREE proved largely ineffective against BADFSS.

5. Conclusion

The discovery of BADFSS highlights a critical security gap in Federated Self-Supervised Learning. Current federated defense mechanisms, designed for supervised settings, are inadequate for the subtle, representation-based shifts introduced by attention alignment. This work serves as a call to action for the research community to develop "backdoor-aware" aggregation methods capable of securing decentralized, unlabeled learning frameworks.

Summary updated

Abstract

Self-supervised learning (SSL) is capable of learning remarkable representations from centrally available data. Recent works further implement federated learning with SSL to learn from rapidly growing decentralized unlabeled images (e.g., from cameras and phones), often resulting from privacy constraints. Extensive attention has been paid to designing new frameworks or methods that achieve better performance for the SSL-based FL. However, such an effort has not yet taken the security of SSL-based FL into consideration. We aim to explore backdoor attacks in the context of SSL-based FL via an in-depth empirical study. In this paper, we propose a novel backdoor attack BADFSS against SSL-based FL. First, BADFSS learns a backdoored encoder via supervised contrastive learning on poison datasets constructed based on local datasets. Then, BADFSS employs attention alignment to enhance the backdoor effect and maintain the consistency between backdoored and global encoders. Moreover, we perform empirical evaluations of the proposed backdoor attacks on four datasets and compared BADFSS with three existing backdoor attacks that are transferred into federated self-supervised learning. The experiments demonstrate that BADFSS outperforms baseline methods and is effective under various settings.
BibTeX
@inproceedings{zhang2024badfss,
  author = {Jiale Zhang and Chengcheng Zhu and Di Wu and Xiaobing Sun and Jianming Yong and Guodong Long},
  title = {{BADFSS: Backdoor Attacks on Federated Self-Supervised Learning}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
  url = {https://openreview.net/forum?id=NOJ3qj5FFD},
}
Figure