Robust Self-Weighted Multi-View Projection Clustering

Beilei Wang, Yun Xiao, Zhihui Li, Xuanhong Wang, Xiaojiang Chen, Dingyi Fang

Abstract

Robust Self-Weighted Multi-View Projection Clustering (RSwMPC) is a novel framework designed to cluster complex, high-dimensional, and noisy data from multiple sources. By integrating dimensionality reduction, noise suppression via the $\ell_{2,1}$-norm, and automated view weighting into a single optimization step, RSwMPC produces a high-quality similarity graph with a clear block-diagonal structure. This allows for direct cluster identification without the need for post-processing steps like k-means. Experimental results on synthetic and real-world datasets demonstrate that RSwMPC consistently outperforms state-of-the-art methods in accuracy, robustness, and purity.

1. Introduction

In real-world applications, data is frequently collected from multiple "views"—for instance, a video sequence can be represented by audio signals, pixel intensities, and motion vectors. While multi-view data provides a more comprehensive description of the underlying patterns, it introduces three significant challenges for machine learning: the curse of dimensionality, the presence of noise and outliers, and the difficulty of effectively integrating views with varying levels of informativeness. Traditional clustering methods often address these issues in isolation, which can lead to suboptimal results. This paper presents RSwMPC, a unified framework that simultaneously performs subspace learning, robust feature selection, and automated view weighting to learn a structured similarity graph for direct clustering.

2. Methodology

The RSwMPC framework is defined by a unified objective function that optimizes several components concurrently:

  • Subspace Learning and Dimensionality Reduction: The algorithm learns a projection matrix ($W$) for each view, mapping high-dimensional data into a cleaner, lower-dimensional subspace where clusters are more easily separable.
  • Robustness via $\ell_{2,1}$-norm: To mitigate the impact of noise and outliers, RSwMPC employs the $\ell_{2,1}$-norm on the projection matrix. This promotes row-sparsity, effectively performing feature selection by identifying the most relevant dimensions and suppressing noisy ones.
  • Automated Self-Weighting: A parameter-free strategy is introduced to automatically determine the importance ($\alpha$) of each view. Views that yield a clearer clustering structure are assigned higher weights, eliminating the need for manual weight tuning.
  • Structured Graph Learning: By imposing a Laplacian rank constraint on the similarity matrix ($S$), the model ensures the resulting graph contains exactly $c$ connected components, corresponding to the number of clusters. This allows for direct label assignment from the graph structure.

3. Experimental Evaluation

The performance of RSwMPC was evaluated against several state-of-the-art methods using both synthetic and real-world datasets, including MSRC-v1, Caltech101 (subsets 7 and 20), NUS-WIDE, ORL Face, and Handwritten Numerals (HW).

3.1 Key Results

RSwMPC demonstrated superior performance across all benchmarks. Notably, on the Caltech101-7 and Caltech101-20 datasets, it achieved accuracy improvements of approximately 9.5% and 9.9% over the second-best methods. On the ORL Face dataset, the algorithm achieved a perfect score of 1.0 for Accuracy, NMI, and Purity.

Comparison of similarity matrices learned by different methods. RSwMPC (g) shows the clearest block-diagonal structure compared to individual views (a, b) and other multi-view methods (c-f).

4. Discussion

A key advantage of RSwMPC is its ability to learn a "clean" similarity graph. As shown in the figure above, the similarity matrix produced by RSwMPC exhibits a distinct block-diagonal structure, where clusters are separated into independent connected components. This eliminates the uncertainty often introduced by post-processing steps like k-means. Furthermore, noise-injection tests on the HW dataset confirmed the model's robustness, as its performance remained stable while other methods' accuracy declined significantly with increased noise levels.

5. Conclusion

RSwMPC provides a robust and automated solution for multi-view clustering. By unifying projection, noise suppression, and view weighting, it effectively handles high-dimensional and noisy data. Future work may focus on reducing the $O(n^2)$ computational complexity to better accommodate large-scale datasets and exploring semi-supervised extensions to further guide the subspace learning process.

Summary updated

Abstract

Many real-world applications involve data collected from different views and with high data dimensionality. Furthermore, multi-view data always has unavoidable noise. Clustering on this kind of high-dimensional and noisy multi-view data remains a challenge due to the curse of dimensionality and ineffective de-noising and integration of multiple views. Aiming at this problem, in this paper, we propose a Robust Self-weighted Multi-view Projection Clustering (RSwMPC) based on ℓ2,1-norm, which can simultaneously reduce dimensionality, suppress noise and learn local structure graph. Then the obtained optimal graph can be directly used for clustering while no further processing is required. In addition, a new method is introduced to automatically learn the optimal weight of each view with no need to generate additional parameters to adjust the weight. Extensive experimental results on different synthetic datasets and real-world datasets demonstrate that the proposed algorithm outperforms other state-of-the-art methods on clustering performance and robustness.
BibTeX
@inproceedings{wang2020robust,
  author = {Beilei Wang and Yun Xiao and Zhihui Li and Xuanhong Wang and Xiaojiang Chen and Dingyi Fang},
  title = {{Robust Self-Weighted Multi-View Projection Clustering}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i04.6075},
}
Figure