FedCC: Robust Federated Learning against Model Poisoning Attacks

Hyejun Jeong, Ha Min Son, Seo Hu Lee, Jayun Hyun, Tai-Myoung Chung

Abstract

Federated Learning has emerged to cope with raising concerns about privacy breaches in using Machine or Deep Learning models. This new paradigm allows the leverage of deep learning models in a distributed manner, enhancing privacy preservation. However, the server's blindness to local datasets introduces its vulnerability to model poisoning attacks and data heterogeneity, tampering with the global model performance. Numerous works have proposed robust aggregation algorithms and defensive mechanisms, but the approaches are orthogonal to individual attacks or issues. FedCC, the proposed method, provides robust aggregation by comparing the Centered Kernel Alignment of Penultimate Layers Representations. The experiment results on FedCC demonstrate that it mitigates untargeted and targeted model poisoning or backdoor attacks while also being effective in non-Independently and Identically Distributed data environments. By applying FedCC against untargeted attacks, global model accuracy is recovered the most. Against targeted backdoor attacks, FedCC nullified attack confidence while preserving the test accuracy. Most of the experiment results outstand the baseline methods.

Introduction

Federated Learning (FL) enables multiple clients to collaboratively train a global model without sharing their raw data. While this decentralization enhances privacy, it creates a significant "attack surface." Malicious clients can execute model poisoning attacks, either to degrade overall model performance (untargeted) or to inject specific "backdoors" (targeted) that trigger misclassification on specific inputs.

A primary challenge in defending against these attacks is data heterogeneity, or non-IID (Independently and Identically Distributed) data. In real-world scenarios, honest clients often possess unique data distributions. Traditional defenses, such as Krum or Coordinate-wise Median, rely on geometric distances between model weights. In non-IID settings, a legitimate client's update may appear as an outlier simply due to its unique data, leading to "friendly fire" where benign updates are discarded while stealthy attacks remain undetected.

Methodology: The FedCC Framework

FedCC (Federated Centered Kernel Alignment) shifts the defensive focus from raw weight statistics to the structural representations learned by the models. The framework operates through a multi-step process on the server side to filter and aggregate updates.

The FedCC workflow: from local model training to CKA-based clustering and weighted aggregation.

Penultimate Layer Representations (PLRs)

The authors identify the penultimate layer (the second-to-last layer) as the most discriminative for attack detection. This layer captures high-level features immediately preceding the final classification decision. By analyzing these representations, FedCC can distinguish between benign diversity and malicious corruption.

Centered Kernel Alignment (CKA)

Instead of measuring Euclidean or Cosine distances between weight vectors—which are sensitive to scaling and orthogonal transformations—FedCC utilizes Centered Kernel Alignment (CKA). CKA measures the similarity between the global model's PLR and each client's PLR. This metric is robust in non-IID settings because it focuses on the underlying structure of the learned representations rather than the specific values of the parameters.

Clustering and Weighted Aggregation

The server computes CKA values for all clients and employs a K-means algorithm ($K=2$) to group them. Assuming malicious clients are in the minority, the larger cluster is identified as benign. FedCC then applies a layer-wise weighted aggregation. For the penultimate layer, weights are normalized within the benign cluster. For other layers, weights are normalized across all clients to preserve parameter scale while attenuating the influence of suspicious updates.

Experimental Results and Insights

The effectiveness of FedCC was evaluated using Fashion-MNIST, CIFAR-10, and CIFAR-100 datasets under highly skewed non-IID conditions (Dirichlet distribution $\alpha=0.2$).

Empirical Justification for PLRs

A key insight of the research is the empirical proof that the distance between benign and malicious clusters is maximized at the penultimate layer. As shown in the experimental data, the PLR acts as a "tripwire," where the divergence between attack types and benign behavior is most pronounced compared to earlier layers.

Cluster distance across different network layers, demonstrating that the Penultimate Layer Representation (PLR) provides the highest discriminative power for detecting attacks.

Performance Highlights

  • Backdoor Mitigation: FedCC reduced backdoor attack success rates to nearly zero across all datasets, effectively nullifying targeted attacks.
  • Accuracy Recovery: Against untargeted "Fang" attacks, FedCC improved global model accuracy by an average of 65.5% over standard robust aggregation baselines.
  • Non-IID Resilience: In complex CIFAR-100 scenarios, FedCC maintained stable test accuracy (14-16%) while traditional methods like Krum often collapsed to near-zero performance.
  • Resistance to Adaptive Attacks: The method remained robust against Distributed Backdoor Attacks (DBA), where the attack trigger is split across multiple clients to evade simple outlier detection.

Conclusion

FedCC provides a robust defense for Federated Learning by leveraging the representational similarity of the penultimate layer. By moving beyond weight-based metrics to CKA-based representation analysis, it successfully navigates the complexities of non-IID data and sophisticated poisoning attacks. Future work will explore the generalizability of FedCC to larger architectures like Vision Transformers and address the computational overhead associated with kernel matrix calculations in large-scale deployments.

Summary updated

Abstract

Federated Learning has emerged to cope with raising concerns about privacy breaches in using Machine or Deep Learning models. This new paradigm allows the leverage of deep learning models in a distributed manner, enhancing privacy preservation. However, the server's blindness to local datasets introduces its vulnerability to model poisoning attacks and data heterogeneity, tampering with the global model performance. Numerous works have proposed robust aggregation algorithms and defensive mechanisms, but the approaches are orthogonal to individual attacks or issues. FedCC, the proposed method, provides robust aggregation by comparing the Centered Kernel Alignment of Penultimate Layers Representations. The experiment results on FedCC demonstrate that it mitigates untargeted and targeted model poisoning or backdoor attacks while also being effective in non-Independently and Identically Distributed data environments. By applying FedCC against untargeted attacks, global model accuracy is recovered the most. Against targeted backdoor attacks, FedCC nullified attack confidence while preserving the test accuracy. Most of the experiment results outstand the baseline methods.
BibTeX
@misc{jeong2022fedcc,
  author = {Hyejun Jeong and Ha Min Son and Seo Hu Lee and Jayun Hyun and Tai-Myoung Chung},
  title = {{FedCC: Robust Federated Learning against Model Poisoning Attacks}},
  year = {2022},
  eprint = {2212.01976},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2212.01976},
}
Figure