Federated Learning (FL) has emerged as a critical paradigm for training machine learning models across distributed devices without centralizing private data. However, FL environments face a dual challenge: high vulnerability to adversarial attacks and performance degradation due to non-IID (Independent and Identically Distributed) data. Standard Adversarial Training (AT), while effective in centralized settings, often fails in FL because local models become heavily biased toward their specific, heterogeneous data distributions.
To address these issues, the framework FatCC (Federated Adversarial Training with Logit Calibration and Feature Contrast) was developed. FatCC aims to maintain high accuracy on clean data while significantly bolstering resilience against adversarial perturbations, even when data is unevenly distributed across the network.
The FatCC Framework
FatCC modifies the standard Federated Adversarial Training process by introducing two key mechanisms designed to mitigate the effects of data heterogeneity and label skewness:
- Local Logit Calibration: To combat label skew (where a client may only have access to a subset of classes), FatCC adjusts the "logits"—the raw scores produced by the model before the final prediction. By applying a modulating factor based on local class frequency, the model is encouraged to pay more attention to minority classes. This prevents the local model from over-fitting to the dominant labels present on a single device.
- Global Feature Contrast: The central server maintains a set of "global features," which are averaged representations of each class collected from across the entire network. During local training, clients employ a contrastive loss term that pulls local features toward the corresponding global class representation (positive pairs) and pushes them away from representations of different classes (negative pairs). This provides a consistent global signal, ensuring that internal representations remain aligned with the global truth regardless of local data bias.
Experimental Evaluation
The researchers evaluated FatCC using three standard computer vision benchmarks: MNIST, Fashion-MNIST, and CIFAR-10. To simulate realistic edge computing scenarios, they utilized a Dirichlet distribution to create varying levels of data heterogeneity among clients. FatCC was compared against several baselines, including FedPGD, FedALP, and FedALC.
Key Results
FatCC consistently outperformed existing federated defense methods in both Clean Accuracy (CA) and Robust Accuracy (RA). For instance, on the Fashion-MNIST dataset under non-IID conditions, FatCC achieved 68.58% CA and 46.82% RA (under the rigorous AutoAttack benchmark). In contrast, the next best method, FedALC, achieved only 65.46% CA and 37.84% RA.
As shown in the comparative data, FatCC effectively bridges the gap between accuracy and robustness. While standard FAT often suffers a significant drop in clean accuracy to gain marginal robustness, FatCC recovers much of this lost performance by addressing the underlying data heterogeneity.
Discussion and Future Work
A notable advantage of FatCC is its privacy-preserving nature. The global features used for contrastive learning are low-dimensional, doubly-averaged vectors, making them significantly more secure than sharing raw data or high-dimensional gradients. Furthermore, the method demonstrates superior communication efficiency, converging to higher accuracy levels in fewer rounds than its competitors.
Future research directions include automating the tuning of the logit calibration hyperparameters ($\alpha$ and $\beta$) to make the system more "plug-and-play." Additionally, while this work focuses on label skewness, future iterations could explore feature skewness, where the visual characteristics of the same class vary across different devices (e.g., varying lighting conditions). Testing on larger-scale datasets like ImageNet will also be vital to validating the framework's scalability.
Summary updated