Abstract
Federated learning (FL) enables participants to store data locally while collaborating in training, yet it remains vulnerable to privacy attacks, such as data reconstruction. Existing differential privacy (DP) technologies inject noise dynamically into the training process to mitigate the impact of excessive noise. However, this dynamic scheduling is often grounded in factors indirectly related to privacy, making it difficult to clearly explain the intricate relationship between dynamic noise adjustments and privacy requirements. To address this issue, we propose FedSDP, a novel and explainable DP-based privacy protection mechanism that guides noise injection based on privacy contribution. Specifically, FedSDP leverages Shapley values to assess the contribution of private attributes to local model training and dynamically adjusts the amount of noise injected accordingly. By providing theoretical insights into the injection of varying scales of noise into local training, FedSDP enhances interpretability. Extensive experiments demonstrate that FedSDP can achieve a superior balance between privacy preservation and model performance, surpassing state-of-the-art (SOTA) solutions.
Introduction
Federated Learning (FL) has emerged as a powerful paradigm for decentralized machine learning, allowing multiple clients to collaboratively train a global model without sharing raw data. Despite its privacy-preserving nature, FL is susceptible to "Honest-But-Curious" adversaries who can perform data reconstruction attacks, reverse-engineering private training samples from shared model updates.
Differential Privacy (DP) is the standard defense against such attacks, typically involving the injection of random noise into model updates. However, DP faces a critical "Privacy-Utility Trade-off": excessive noise degrades model accuracy, while insufficient noise leaves data vulnerable. Current dynamic noise methods often rely on indirect signals like weight changes, which lack a clear explanation of why specific noise levels are chosen.
The FedSDP Framework
FedSDP (Federated Shapley Differential Privacy) addresses these challenges by making noise injection explainable and efficient through the use of Shapley values from cooperative game theory. The framework consists of two primary modules:
- Privacy Estimation (PE): This module quantifies the "marginal benefit" of private attributes. Each client distinguishes between "general" and "private" attributes in their local data. By comparing model performance with and without these attributes, the PE module calculates a privacy contribution ratio, indicating how much private data influences the model in a given round.
- Noise Injection (NI): Instead of static noise, the NI module dynamically scales Gaussian noise based on the privacy contribution ratio. Noise is increased when private attributes are highly influential (typically early in training) and reduced as their contribution wanes.
Experimental Evaluation
The authors evaluated FedSDP on two standard benchmarks: CIFAR-10 (using ResNet-18) and FEMNIST (using a CNN). To test robustness, they simulated data reconstruction attacks using an Auxiliary Classifier GAN (ACGAN).
Key Results
- Superior Accuracy: On CIFAR-10 with a strict privacy budget ($\epsilon = 0.1$), FedSDP achieved 61.19% accuracy, significantly outperforming the NbAFL baseline (42.21%).
- Optimized Trade-offs: FedSDP consistently maintained higher accuracy than other state-of-the-art dynamic DP methods across various privacy budgets.
- Robust Defense: Qualitative analysis demonstrates that FedSDP effectively thwarts reconstruction attacks. While unprotected models (FedAvg) allow attackers to recover recognizable images, FedSDP successfully obfuscates these features to a level comparable with much noisier, less accurate methods.
Discussion and Future Work
FedSDP moves DP from a heuristic-based approach to a quantitative, explainable indicator. By targeting noise injection specifically when private attributes are most influential, it avoids the "blanket" noise application that typically hinders model performance.
However, challenges remain. Calculating exact Shapley values is computationally intensive, and the current method relies on clear separation between private and general attributes. Future research will focus on optimizing computational efficiency and extending the framework to more complex, non-IID data distributions.
Summary updated