Abstract
Federated Learning (FL) is designed to protect data privacy by keeping raw data on local devices. However, sharing model updates can still leak sensitive information, and models often inherit or amplify biases against minority groups. This paper explores the "trilemma" of balancing privacy, fairness, and utility. We introduce FedLDP, a framework that integrates local fairness constraints with Local Differential Privacy (LDP). Our findings reveal a counter-intuitive phenomenon: while privacy-preserving noise typically degrades fairness, it can unexpectedly enhance model accuracy by assisting optimizers in escaping local optima.
Introduction
Federated Learning faces two critical ethical challenges. First, Privacy Leakage: gradients shared during training can inadvertently reveal sensitive user data. Second, Algorithmic Bias: models may perform poorly for underrepresented subgroups. These goals often conflict; strict Differential Privacy (DP) can disproportionately impact minority group accuracy, while achieving fairness often requires sharing more detailed statistics, increasing privacy risks. This work maps the intrinsic equilibrium between these three competing forces.
Methodology
The proposed FedLDP framework manages these trade-offs through a two-stage process:
- Fairness Stage (Fair-SGD): Fairness is treated as a constrained optimization problem using Lagrangian multipliers to create a Min-Max objective. This ensures the model adheres to metrics such as Demographic Parity (DemP), Equalized Odds (EO), and Disparate Impact (DI).
- Privacy Stage: To protect the fair gradients, the authors apply Local Differential Privacy (LDP) using a Gaussian Mechanism, adding noise before central aggregation.
As shown in Table 1, the proposed method is categorized as a "Type II" trade-off, focusing on the interaction between privacy and fairness. While it shares this categorization with recent works like [LGR23b] and [TFVH21], it incorporates a broader set of fairness metrics including Disparate Impact (DI).
Experimental Setup
The framework was evaluated using the Adult (Census Income) Dataset, containing 48,842 records. The task was to predict whether an individual's annual income exceeds $50,000, with Race (White vs. Black) serving as the sensitive attribute for fairness evaluation.
Key Results and Insights
The study reveals several critical insights into the privacy-fairness-utility trilemma:
- Privacy as a "Constraint Breaker": Privacy noise acts as a disruptor to fairness. Since fairness is implemented as mathematical constraints, the random noise from DP "breaks" these constraints, leading to higher fairness errors.
- The Utility Paradox: Surprisingly, adding privacy noise increased test accuracy for some clients. For instance, Client 1’s accuracy for the "Black" subgroup rose from 32.20% (no privacy) to 66.63% (with privacy). The authors hypothesize that noise allows the optimizer to "jump out" of local optima, finding better global solutions.
- Fairness Degradation: While EO and DemP errors converged toward zero without privacy, they failed to converge once noise was introduced, confirming that privacy protections directly interfere with fairness goals.
Conclusion and Future Work
The research concludes that there is no "free lunch" in FL; maintaining privacy, fairness, and utility simultaneously requires a carefully tuned equilibrium. Future work should explore these trade-offs in high-dimensional datasets (e.g., images), calibrate noise levels to find the utility "tipping point," and examine decentralized FL environments where trust models may differ.
Summary updated