Abstract
Federated learning has emerged as a powerful paradigm for privacy-preserving machine learning, yet its application to unsupervised tasks like clustering remains challenging due to data heterogeneity. Standard federated clustering methods often produce a single global model that fails to represent diverse local data distributions, leading to performance inequities among participants. We present p-FClus, a novel personalized federated clustering algorithm. By enabling clients to fine-tune global cluster centers to their local data in a single communication round, p-FClus significantly reduces individual clustering costs and ensures fairness. Experimental results across synthetic and real-world datasets demonstrate that p-FClus consistently outperforms state-of-the-art methods in terms of mean cost, cost stability, and worst-case performance.
Introduction
The proliferation of edge devices has led to an explosion of unlabeled data. While Federated Learning (FL) offers a path to utilize this data without compromising privacy, the majority of FL research focuses on supervised learning. Federated Unsupervised Learning (FedUL), particularly clustering, faces unique hurdles. The primary challenge is data heterogeneity: local data distributions often vary significantly across clients. For instance, in a clustering task with nine true underlying distributions, one client might only possess data from three, while another possesses all nine.
Existing State-of-the-Art (SOTA) federated clustering methods, such as k-FED or MFC, typically aim for a single global consensus. However, a "one-size-fits-all" model often results in high "clustering costs" for clients whose local data deviates from the global average. This lack of fairness can discourage participation in decentralized systems. Furthermore, clustering lacks the inherent synchronization found in supervised learning (e.g., fixed weight ordering), making the direct extension of personalization techniques non-trivial.
Methodology: The p-FClus Algorithm
To address these challenges, we propose p-FClus (personalized Federated Clustering). The algorithm introduces a personalization step that allows clients to adapt global knowledge to their specific local contexts. The process is structured into three phases:
- Phase 1: Client Initialization: Each client independently executes a local clustering algorithm (e.g., $k$-means or $k$-medoids) on its private data to obtain a set of local cluster centers.
- Phase 2: Server Aggregation: Clients transmit these local centers to a central server. The server then performs a "smart procedure" to find the best global centers that capture the union of all true distributions represented across the client pool. These global centers are then broadcast back to all clients.
- Phase 3: Client Personalization: This is the core innovation of p-FClus. Instead of adopting the global centers directly, each client fine-tunes them using a mathematically supported personalization procedure. This involves a novel objective function that balances minimizing local clustering cost with a regularization term that prevents the personalized centers from straying too far from the global centers.
Notably, p-FClus achieves this personalization in just one communication round, making it highly efficient for resource-constrained edge environments. It is also compatible with any finite $\ell$-norm, supporting various clustering objectives.
Experimental Evaluation
The efficacy of p-FClus was validated against a diverse set of benchmarks, including synthetic datasets with varying cluster overlap and extrinsic real-world datasets such as Adult, Bank, Diabetes, and FMNIST. The researchers also utilized intrinsic federated datasets like FEMNIST (500 writers) and WISDM (36 clients using sensor data) to test performance in naturally decentralized settings.
Key Results
Experimental results consistently show that p-FClus provides superior performance and fairness compared to SOTA baselines:
- Reduced Clustering Cost: p-FClus achieves a lower mean cost per data point. On the WISDM dataset, the improvement was as high as five orders of magnitude compared to k-FED and MFC.
- Enhanced Fairness: The "cost deviation" ($\sigma$)—representing the performance gap between clients—is significantly lower and remains stable even as data heterogeneity increases.
- Minimized Worst-Case Risk: The maximum cost suffered by any individual client is minimized, ensuring that no participant is unfairly penalized by the global model.
Discussion and Future Directions
The success of p-FClus highlights that personalization is a critical tool for achieving fairness in federated unsupervised learning. By bypassing the synchronization problem through point-wise updates and allowing local adaptation, p-FClus bridges the gap between diverse data distributions.
Future research will focus on enhancing the robustness of p-FClus against malicious clients and strategic data manipulation. Additionally, the development of "federated unlearning" techniques—allowing the removal of specific client contributions without full retraining—remains a vital area for ensuring long-term privacy and compliance in decentralized systems.
Summary updated