Multi-view Clustering with the Cooperation of Visible and Hidden Views

Zhaohong Deng, Ruixiu Liu, Te Zhang, Peng Xu, Kup‐Sze Choi, Bin Qin, Shitong Wang

Abstract

Multi-view data clustering is a critical task in modern machine learning, yet existing algorithms often fail to fully exploit the available information. Many methods focus exclusively on the cooperation of visible views in their original spaces or solely on hidden information between views. This paper introduces MV-Co-VH, a novel multi-view clustering algorithm that facilitates cooperation between visible and hidden views. By projecting multiple views into a common hidden space using non-negative matrix factorization (NMF) and implementing collaborative learning, MV-Co-VH effectively captures both the unique "otherness" of individual views and the "consistency" of shared information. Experimental results on diverse datasets demonstrate that MV-Co-VH achieves state-of-the-art performance.

Introduction

In real-world scenarios, data is frequently represented through multiple views. For instance, an image can be described by its color histogram and its texture features. Multi-view clustering aims to group these data points by leveraging the complementary information across different views. However, a significant challenge lies in balancing the "otherness" (view-specific details) and "consistency" (shared underlying structure).

Existing approaches typically fall into two categories:

  1. Visible-focused methods cluster views independently and then fuse them, often missing the shared latent structure.
  2. Hidden-focused methods project views into a single latent subspace, which can discard valuable fine-grained details present in the original views.

The MV-Co-VH Framework

The proposed MV-Co-VH algorithm bridges this gap by forcing visible and hidden information to work in tandem during the clustering process.

The proposed MV-Co-VH framework illustrating the cooperation between visible views and the shared hidden view.

The algorithm proceeds in two primary stages:

1. Shared Hidden View Extraction

Using a modified Non-negative Matrix Factorization (NMF) strategy termed SHD-NMF, the algorithm projects all visible views into a common hidden space. A key innovation here is an adaptive weighting mechanism that assigns weights to each view based on its fit to the shared representation, effectively mitigating the impact of noisy views.

2. Collaborative Clustering

Once the hidden view is extracted, the clustering is performed by minimizing a joint objective function that accounts for cluster dispersion in both the visible and hidden views. This is controlled by a collaborative learning coefficient, $\beta$, which acts as a dial to balance the influence of the two types of views. Additionally, entropy regularization is employed to automatically learn the importance weights for each visible view.

Experimental Evaluation

The authors conducted extensive experiments on eight benchmark datasets, including UCI datasets (Multiple Features, Image Segmentation, Dermatology, Forest Type, Water Treatment Plant), image datasets (Caltech, Corel), and text datasets (Reuters).

Performance Metrics

The algorithm was evaluated using three key indices: Normalized Mutual Information (NMI), Rand Index (RI), and Precision.

Clustering performance comparison of MV-Co-VH against state-of-the-art methods across various indices.

Key Results and Discussion

As shown in the experimental results, MV-Co-VH consistently outperformed seven other methods, including Fuzzy C-Means (FCM), Collaborative Fuzzy K-Means (Co-FKM), and Multi-view NMF (MultiNMF).

  • Superior Accuracy: MV-Co-VH achieved the highest scores across all metrics for the tested datasets. For example, on the Caltech dataset, it reached an NMI of 0.9037, significantly higher than its competitors.
  • Synergy of Views: The study found that the best performance occurred when both visible and hidden views were utilized (i.e., when $\beta$ was between 0 and 1), confirming that the hidden view acts as a vital stabilizer for the potentially noisy visible views.
  • Statistical Significance: Post-hoc analysis confirmed that the performance improvements were statistically significant and not due to random chance.

Conclusion

The MV-Co-VH algorithm represents a significant step forward in multi-view clustering by effectively integrating visible and hidden information. By capturing both the specific details of individual views and the shared essence of the data, it provides a more robust and accurate clustering solution. Future work will focus on automating hyperparameter selection to enhance the algorithm's usability across broader applications.

Summary updated

Abstract

Multi-view data are becoming common in real-world modeling tasks and many multi-view data clustering algorithms have thus been proposed. The existing algorithms usually focus on the cooperation of different views in the original space but neglect the influence of the hidden information among these different visible views, or they only consider the hidden information between the views. The algorithms are therefore not efficient since the available information is not fully excavated, particularly the otherness information in different views and the consistency information between them. In practice, the otherness and consistency information in multi-view data are both very useful for effective clustering analyses. In this study, a Multi-View clustering algorithm developed with the Cooperation of Visible and Hidden views, i.e., MV-Co-VH, is proposed. The MV-Co-VH algorithm first projects the multiple views from different visible spaces to the common hidden space by using the non-negative matrix factorization (NMF) strategy to obtain the common hidden view data. Collaborative learning is then implemented in the clustering procedure based on the visible views and the shared hidden view. The results of extensive experiments on UCI multi-view datasets and real-world image multi-view datasets show that the clustering performance of the proposed algorithm is competitive with or even better than that of the existing algorithms.
BibTeX
@misc{deng2019multi,
  author = {Zhaohong Deng and Ruixiu Liu and Te Zhang and Peng Xu and Kup‐Sze Choi and Bin Qin and Shitong Wang},
  title = {{Multi-view Clustering with the Cooperation of Visible and Hidden Views}},
  year = {2019},
  eprint = {1908.04766},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1908.04766},
}
Figure