Hyper-Laplacian Regularized Concept Factorization in Low-Rank Tensor Space for Multi-View Clustering

Zixiao Yu, Lele Fu, Yongyong Chen, Zhiling Cai, Guoqing ChaoIEEE Trans. Emerg. Top. Comput. Intell 2023

TL;DR

This paper introduces HLRCF, a novel multi-view clustering method that combines concept factorization with hyper-Laplacian regularization and a self-weighted tensor Schatten p-norm. It tackles the limitations of existing methods by efficiently capturing high-order local structures and inter-view correlations, leading to superior clustering performance with reduced computational complexity.

What Problem Does This Paper Address?

Multi-view clustering aims to group data points by leveraging information from multiple sources (views). While tensor-based methods have shown promise in capturing complex relationships, they often suffer from high computational complexity and can struggle to effectively model non-linear local structures. Existing approaches also tend to treat all singular values equally in tensor decomposition, potentially overlooking the varying importance of structural information they encode. This paper addresses these challenges by proposing a method that is both computationally efficient and better equipped to capture intricate data structures.

Methods

The core of HLRCF lies in three key innovations:

  • Concept Factorization (CF) with Hyper-Laplacian Regularization: CF is used to extract latent cluster-wise representations from each view. To capture high-order local structures, a hypergraph Laplacian regularizer is employed, which goes beyond traditional graph structures to model more complex neighborhood relationships.
  • Self-Weighted Tensor Schatten p-norm (SWTSP-norm): Instead of the standard tensor nuclear norm (TNN), which treats all singular values equally, HLRCF introduces a novel SWTSP-norm. This norm adaptively assigns weights to singular values, allowing the model to better leverage the varying importance of structural information encoded in different singular values.
  • Low-Rank Tensor Learning: The cluster-wise representations from all views are stacked into a tensor. This tensor is then optimized using the SWTSP-norm to capture high-order correlations across views in a low-rank tensor space. This tensor-based approach significantly reduces time and space complexity compared to traditional matrix-based methods.

Datasets

The proposed HLRCF method was evaluated on eight publicly available real-world multi-view datasets:

  • BBCsports
  • COIL20
  • NUS-WIDE
  • UCI
  • Caltech101-20
  • GRAZ02
  • Scene15
  • 100leaves

Key Insights and Ideas

  • Hypergraphs for Higher-Order Structure: The use of hypergraphs for regularization is crucial for capturing complex, multi-way relationships between data points, which is a limitation of standard graph-based methods.
  • Adaptive Singular Value Weighting is Key: The SWTSP-norm's ability to adaptively weight singular values is a significant improvement over TNN, allowing for a more nuanced and effective low-rank approximation of tensors. This is particularly important for capturing critical structural information.
  • Tensorization for Efficiency: By forming a tensor of view-specific representations and optimizing it in a low-rank tensor space, the method achieves substantial reductions in both time and space complexity, making it more scalable.
  • Concept Factorization as an Efficient Building Block: CF's ability to project data into a low-dimensional subspace makes it an efficient starting point for extracting representations, contributing to the overall computational efficiency of HLRCF.

Key Results

  • Superior Clustering Performance: HLRCF consistently outperformed fifteen other state-of-the-art multi-view clustering methods on five out of eight benchmark datasets, achieving the best results. It also showed significant improvements over the second-best method (HLR-M²VS) on the UCI and GRAZ02 datasets.
  • Efficiency Gains: The paper highlights that HLRCF achieves a significant reduction in time complexity compared to other tensor-based methods, making it highly competitive. The low-rank tensor optimization with SWTSP-norm is noted as being particularly efficient.
  • Ablation Studies Confirm Module Importance: Ablation experiments demonstrated that both the hyper-Laplacian regularization and the SWTSP-norm are critical for HLRCF's strong performance. Removing either module led to a noticeable drop in clustering quality.

Limitations and Future Work

  • Parameter Sensitivity: While HLRCF shows good performance, the paper notes that parameters like $\lambda$ and $\alpha$ have a significant effect on clustering results, requiring careful tuning. The authors aim to ensure HLRCF remains relatively insensitive to parameter variations in future work.
  • Extensions of Concept Factorization: The authors plan to investigate extensions of concept factorization, including adaptive neighbor selection for the hyper-Laplacian regularizer, to enhance the method's capability in more complex scenarios.

Summary updated

BibTeX
@article{yu2023hyper,
  author = {Zixiao Yu and Lele Fu and Yongyong Chen and Zhiling Cai and Guoqing Chao},
  title = {{Hyper-Laplacian Regularized Concept Factorization in Low-Rank Tensor Space for Multi-View Clustering}},
  journal = {IEEE Trans. Emerg. Top. Comput. Intell.},
  year = {2023},
}
Figure