Scalable Federated One-Step Multi-View Clustering with Tensorized Regularization

Wei Feng, Danting Liu, Qianqian Wang, Wenqi Liang, Zheng YanAAAI 2025 · DOI

TL;DR

This paper introduces SFOMVC-TR, a novel federated multi-view clustering (FMVC) method that addresses the challenges of data privacy and scalability in distributed environments. By leveraging anchor graphs and a one-step clustering approach with tensorized regularization, SFOMVC-TR achieves efficient and effective clustering without requiring clients to share raw data.

What Problem Does This Paper Address?

Multi-view clustering (MVC) methods are powerful for analyzing data from different perspectives, but their application in real-world scenarios is hindered by data privacy concerns. When data is distributed across different organizations, traditional centralized MVC approaches are not feasible. Federated Multi-View Clustering (FMVC) aims to solve this by enabling collaborative model training without data sharing. However, existing FMVC methods often require post-processing to derive cluster labels and struggle to effectively capture the complementary and consistent information across multiple views. This can lead to suboptimal performance and redundancy. The paper tackles this by proposing a method that performs clustering in a single step at each client and uses a centralized server to fuse information, all while preserving privacy.

Methods

The core of SFOMVC-TR lies in its innovative approach to federated learning for multi-view clustering:

  • Anchor Graphs: Instead of using raw multi-view data, SFOMVC-TR utilizes "anchor graphs." These graphs are derived from a selected subset of data points (anchors), significantly reducing the dimensionality and computational complexity, thus improving scalability.
  • One-Step Clustering with Lp,q Regularization: Each client performs clustering in a single step. This is achieved by employing an $L_{p,q}$ sparse regularization on the projection matrix. This regularization helps to remove feature redundancy and directly projects the anchor graph into clustering assignments, eliminating the need for separate feature extraction and clustering stages.
  • Tensorized Regularization for Global Consistency: A centralized server plays a crucial role in fusing the local clustering results from all clients. It uses tensor learning to capture consistent and complementary information across these local results. By minimizing the tensor Schatten-p norm of the fused results, the server enforces consistency among the clients' clustering assignments.
  • Federated Optimization Framework: The entire process is orchestrated by a federated optimization framework. This framework supports collaborative training while ensuring privacy. Clients train local models, share their clustering assignments and local losses (not raw data) with the server, and the server aggregates this information to guide the global model. Adaptive weights are used to balance the contributions of each client and the server.

Datasets

The paper evaluates SFOMVC-TR on eight public multi-view datasets:

  • ORL
  • Yale
  • BBCSport
  • 3-sources
  • WebKB
  • Uci-Digit
  • NGs
  • Reuters

The authors note that for the $L_{p,q}$ regularization, they fix $p=2$ to leverage an analytical solution for $0 < q < 1$.

Key Insights and Ideas

  • Anchor Graphs for Scalability: The shift from raw data to anchor graphs is a key innovation for handling high-dimensional data in a federated setting, making the method more scalable and computationally efficient.
  • One-Step Clustering Efficiency: Eliminating the multi-step process of feature extraction followed by clustering simplifies the pipeline and potentially reduces error propagation.
  • Tensor Learning for Global Cohesion: Using tensor learning at the server level is a sophisticated way to fuse information from disparate clients, ensuring that the global clustering result reflects the collective understanding of the data across all views and entities.
  • Privacy-Preserving by Design: The framework is designed to avoid sharing raw data or anchor graphs, with only clustering assignments and local losses being transmitted, which significantly enhances privacy.

Key Results

  • Superior Performance: SFOMVC-TR outperforms other federated multi-view clustering methods, demonstrating comparable performance to centralized MVC methods on several datasets.
  • High Accuracy: On the ORL dataset, SFOMVC-TR achieved 100% in Accuracy (ACC), Normalized Mutual Information (NMI), and Purity (PUR).
  • Robustness on Large Datasets: Even on the large Reuters dataset, where many algorithms struggled due to memory constraints, SFOMVC-TR achieved 72.79% ACC, 64.51% NMI, and 77.22% PUR.
  • Ablation Studies Confirm Component Importance: Ablation experiments showed that both the sparse constraint ($L_{p,q}$ regularization) and the tensor regularization are crucial for achieving optimal performance, highlighting the contribution of each component.

Limitations and Future Work

  • Parameter Sensitivity: The paper acknowledges that the performance is sensitive to hyperparameter choices, specifically $\beta$ and $\gamma$. While they provide analysis on their impact, finding optimal values might require further tuning.
  • Anchor Selection Strategy: The paper mentions using a basic anchor selection strategy. Further research could explore more advanced federated anchor selection methods to potentially improve performance and robustness.
  • Generalization of Lp,q: The paper focuses on $p=2$ for the $L_{p,q}$ regularization due to analytical solvability. Exploring the full range of $p$ and $q$ in a federated context could be a future direction.
  • Security Model: The current security model assumes semi-honest clients and server. Investigating robustness against more malicious adversaries could be a valuable extension.

Summary updated

BibTeX
@inproceedings{feng2025scalable,
  author = {Wei Feng and Danting Liu and Qianqian Wang and Wenqi Liang and Zheng Yan},
  title = {{Scalable Federated One-Step Multi-View Clustering with Tensorized Regularization}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i16.33822},
}
Figure