Abstract
In domains where data are sensitive or private, there is great value in methods that can learn in a distributed manner without the data ever leaving the local devices. In light of this need, federated learning has emerged as a popular training paradigm. However, many federated learning approaches trade transmitting data for communicating updated weight parameters for each local device. Therefore, a successful breach that would have otherwise directly compromised the data instead grants whitebox access to the local model, which opens the door to a number of attacks, including exposing the very data federated learning seeks to protect. Additionally, in distributed scenarios, individual client devices commonly exhibit high statistical heterogeneity. Many common federated approaches learn a single global model; while this may do well on average, performance degrades when the i.i.d. assumption is violated, underfitting individuals further from the mean, and raising questions of fairness. To address these issues, we propose Weight Anonymized Factorization for Federated Learning (WAFFLe), an approach that combines the Indian Buffet Process with a shared dictionary of weight factors for neural networks. Experiments on MNIST, FashionMNIST, and CIFAR-10 demonstrate WAFFLe's significant improvement to local test performance and fairness while simultaneously providing an extra layer of security.
Introduction
Standard Federated Learning (FL) algorithms, such as FedAvg, typically aim to learn a single global model. However, real-world data is rarely independent and identically distributed (i.i.d.), leading to significant challenges in statistical heterogeneity and security. A "one-size-fits-all" model often performs poorly for users whose data differs from the majority, creating a fairness gap. Furthermore, sharing full model weights with a central server can leak sensitive information via whitebox attacks like membership inference.
WAFFLe (Weight Anonymized Factorization for Federated Learning) addresses these issues by using a shared dictionary of rank-1 weight factors and the Indian Buffet Process (IBP). This framework allows clients to build personalized models while keeping the specific combination of factors private, enhancing both performance and security.
Methods: Personalized Factorization
Instead of learning full weight matrices, WAFFLe learns a dictionary of rank-1 factors. For any given client $i$, the weight matrix is reconstructed as $W_i = W_a \Lambda_i W_b$.
As illustrated in Figure 1, $W_a$ and $W_b$ are global dictionaries shared by all participants, while $\Lambda_i$ is a client-specific diagonal matrix that selects and scales those factors.
Key technical innovations include:
- Indian Buffet Process (IBP): A Bayesian nonparametric prior that encourages sparsity (each client uses few factors) and reuse (clients share common factors), allowing the dictionary to grow dynamically.
- Anonymization through Obfuscation: Clients send updates for the global dictionaries back to the server but never share their selection matrix ($\Lambda_i$). The server knows the "ingredients" but not the "recipe" for any specific client.
- Variational Inference: The model uses "Bayes by Backprop" with differentiable relaxations to learn factor selections locally.
Experimental Results
The authors evaluated WAFFLe on MNIST, FashionMNIST, and CIFAR-10 using a "multimodal" setting to simulate extreme real-world heterogeneity (e.g., assigning majority clients one set of labels and minority clients another).
Accuracy and Fairness
WAFFLe demonstrates superior local accuracy and effectively closes the fairness gap. On CIFAR-10, WAFFLe achieved 66.35% local accuracy, compared to 45.46% for FedAvg and 44.95% for FedProx.
In the MNIST multimodal test, FedAvg exhibited a massive 29.23% performance gap between majority and minority clients. As shown in Figure 2, WAFFLe slashed this gap to just 2.07%, providing nearly uniform performance across all users.
Security and Efficiency
WAFFLe provides an insulating layer of security. In a simulated Membership Inference Attack, the success rate against FedAvg was ~84%, while against WAFFLe, it dropped to ~56%—only slightly better than random guessing. Furthermore, the rank-1 factorization is highly compact, often resulting in fewer total learnable parameters than standard models despite the added personalization.
Conclusion
WAFFLe presents a novel compromise between global and local models by framing personalization as a factor selection problem. By withholding selection indices, it achieves significant gains in accuracy and fairness while thwarting common whitebox attacks. Future work includes scaling this factorization to deeper networks and optimizing the local computational burden of variational inference.
Summary updated