O-Minus Decomposition for Multiview Tensor Subspace Clustering

Yingcong Lu, Yipeng Liu, Zhen Long, Zhangxin Chen, Ce ZhuIEEE Trans. Artif. Intell 2024 · DOI

TL;DR

This paper introduces a novel tensor decomposition method, Tucker-O-Minus Decomposition (TOMD), designed to improve multi-view subspace clustering. By enhancing the capture of global low-rank information and providing a more compact representation, TOMD-MVC significantly outperforms existing methods across various benchmark datasets.

What Problem Does This Paper Address?

Multi-view clustering aims to group data points based on information from multiple sources (views). A common approach is to leverage the self-representation property of data, assuming that each data point can be represented as a linear combination of other data points. This is often modeled using low-rank tensor decomposition, which excels at capturing latent structures in high-dimensional data. However, existing tensor decomposition methods like Tucker decomposition and tensor singular value decomposition (t-SVD) suffer from issues such as unbalanced matricization schemes, rotation sensitivity, and difficulty in capturing correlations between weakly connected modes. These limitations hinder their ability to fully exploit global low-rank information, leading to suboptimal clustering performance.

Methods

The core contribution of this paper is the Tucker-O-Minus Decomposition (TOMD). This novel tensor network architecture builds upon the standard Tucker decomposition by incorporating an "O-minus structure." This structure features a circle with an efficient bridge linking two weakly correlated factors, effectively strengthening their relationships. This is achieved by replacing the core tensor in the Tucker format with this O-minus architecture, resulting in a more balanced structure and enhanced capacity for capturing global low-rank information.

The proposed TOMD is then applied to a multi-view clustering framework, resulting in the TOMD-MVC model. The optimization problem is solved using the Alternating Direction Method of Multipliers (ADMM). The paper also details the TOMD-ALS algorithm for computing the Tucker-O-Minus decomposition itself.

Datasets

The paper evaluates TOMD-MVC on six real-world multi-view datasets:

  • Yale
  • MSRCV1
  • ExtendYaleB
  • ORL
  • Reuters
  • Handwritten

Key Insights and Ideas

  • The O-minus structure is key: By introducing a specific "bridge" mechanism within the tensor network, the authors effectively enhance the capture of global low-rank information, which is crucial for multi-view clustering.
  • Balancing structure and complexity: TOMD aims to achieve a more balanced tensor network structure, improving information capture without incurring prohibitive computational complexity.
  • TOMD-MVC outperforms existing methods: The proposed model consistently achieves superior clustering performance across various metrics, demonstrating the effectiveness of the TOMD framework.
  • Tensor decomposition is powerful for multi-view data: The paper reinforces the idea that leveraging higher-order tensor structures is beneficial for capturing complex correlations present in multi-view datasets.

Key Results

The paper presents extensive experimental results demonstrating the superiority of TOMD-MVC.

  • Image Reconstruction: Experiments on image reconstruction show that TOMD excels at exploring low-rank information, outperforming Tucker-ALS, TuTR-ALS, and Ominus-ALS in terms of storage cost and low-rank information exploitation.
  • Clustering Performance: On six benchmark datasets, TOMD-MVC consistently achieves state-of-the-art performance. For instance, on the Yale dataset, it shows improvements of 1.5% over the second-best method. On the ExtendYaleB dataset, the improvement is at least 30% higher than the second-best method (t-SVD-MSC).
  • Comparison with Tucker-based methods: TOMD-MVC shows significant improvements over 3D and 4D Tucker decomposition-based methods, highlighting the benefits of the O-minus structure and higher-order tensor representation.

Limitations and Future Work

  • Parameter Sensitivity: The paper acknowledges that the performance of TOMD-MVC is influenced by parameters like µ, K, and the tensor ranks. While some analysis is provided, further investigation into optimal parameter selection could be beneficial.
  • Computational Complexity: While the paper analyzes the computational complexity, it is noted that the iterative nature of the algorithm can still be time-consuming, especially for very large datasets.
  • Future Work: The authors suggest that the proposed TOMD framework can be extended to other tensor-based applications beyond clustering, such as tensor completion and dimensionality reduction. Further research into optimizing the TOMD architecture and its application to different data modalities is also implied.

Summary updated

BibTeX
@article{lu2024o,
  author = {Yingcong Lu and Yipeng Liu and Zhen Long and Zhangxin Chen and Ce Zhu},
  title = {{O-Minus Decomposition for Multiview Tensor Subspace Clustering}},
  journal = {IEEE Trans. Artif. Intell.},
  year = {2024},
  doi = {10.1109/tai.2023.3293479},
}
Figure