Multi-view Clustering via Deep Matrix Factorization and Partition Alignment

Chen Zhang, Siwei Wang, Jiyuan Liu, Sihang Zhou, Pei Zhang, Xinwang Liu, En Zhu, Changwang Zhang

Introduction

Multi-view clustering (MVC) aims to group data points by leveraging information from multiple sources or "views"—for instance, a news article represented by both its text and accompanying images. While multi-view data provides a richer context than single-view data, effectively fusing these disparate sources remains a significant challenge. This paper introduces MVC-DMF-PA, a novel framework that integrates deep matrix factorization with a robust "late-fusion" partition alignment strategy to achieve state-of-the-art clustering performance.

The Challenge of Multi-view Clustering

Traditional MVC methods often struggle with three primary issues:

  1. Shallow Representations: Single-layer models frequently fail to capture the complex, hierarchical structures inherent in high-dimensional raw data.
  2. Information Imbalance: Many algorithms prioritize shared information across views while neglecting unique, view-specific features that could enhance cluster discriminability.
  3. The Alignment Problem: In late-fusion approaches, different views may assign inconsistent labels to the same cluster (e.g., View A identifies a group as "Cluster 1" while View B labels it "Cluster 5"). Without a mathematical mechanism to align these assignments, the final fusion is often inaccurate.

Proposed Methodology: MVC-DMF-PA

The MVC-DMF-PA framework addresses these challenges through a unified optimization approach that combines deep feature learning with partition alignment.

The MVC-DMF-PA framework architecture, illustrating the flow from multi-view input to deep feature extraction and final partition fusion.

Deep Semi-NMF

Instead of a single decomposition step, the model employs multiple layers of semi-non-negative matrix factorization (Semi-NMF). This architecture allows the model to learn "hidden" attributes at increasing levels of abstraction, effectively mimicking the hierarchical learning capabilities of deep neural networks.

Partition Alignment and Fusion

To resolve the label-switching problem, the authors introduce a consensus partition matrix ($H$) and a set of rotation/permutation matrices ($W$). These $W$ matrices mathematically "rotate" the cluster indicators of each individual view to align them with a global consensus. This alignment is solved elegantly using Singular Value Decomposition (SVD), avoiding the need for computationally expensive brute-force matching.

Unified Optimization and Weighting

Unlike two-stage methods, MVC-DMF-PA optimizes deep decomposition and partition alignment simultaneously. This ensures that the clustering objective directly guides the feature extraction process. Furthermore, the model adaptively learns weights for each view, automatically emphasizing the most reliable data sources for the final result.

Experimental Evaluation

The researchers evaluated MVC-DMF-PA against 12 state-of-the-art methods using six benchmark datasets:

  • Text: BBC, BBCSport, and Reuters.
  • Image: MSRCV1, ORL (facial recognition), and HW (handwritten digits).

These datasets range from 2 to 5 views and 5 to 40 clusters, providing a comprehensive test of the algorithm's versatility and robustness.

Results and Discussion

MVC-DMF-PA demonstrated superior performance across all metrics, including Accuracy, Normalized Mutual Information (NMI), and Purity.

t-SNE visualization showing the evolution of cluster separation over 60 iterations for the BBC and BBCSport datasets.

Key Findings

  • Performance Gains: The model achieved significant improvements over the second-best methods, notably a 19.85% increase in accuracy on the BBCSport dataset and an 11.68% increase on the BBC dataset.
  • Consistent Ranking: The framework achieved an "Average Rank of 1.0" across all datasets for Accuracy and Purity, indicating it was the top performer in nearly every test case.
  • Depth and Robustness: Ablation studies confirmed that three-layer models consistently outperformed shallower versions, proving that deep information mining is essential for noisy, high-dimensional data.
  • Efficiency: Despite its depth, the model converges rapidly, typically reaching a stable solution within 20 to 60 iterations.

Conclusion

MVC-DMF-PA represents a significant advancement in multi-view clustering by effectively combining deep representation learning with a mathematically rigorous alignment strategy. While the model requires careful hyperparameter tuning and carries a higher computational load than shallow methods, its ability to uncover deep structures and align disparate views makes it a powerful tool for complex data analysis. Future work may focus on automating layer selection and incorporating graph-based local structures to further enhance performance.

Summary updated

BibTeX
@inproceedings{zhang2021multi,
  author = {Chen Zhang and Siwei Wang and Jiyuan Liu and Sihang Zhou and Pei Zhang and Xinwang Liu and En Zhu and Changwang Zhang},
  title = {{Multi-view Clustering via Deep Matrix Factorization and Partition Alignment}},
  booktitle = {ACM Multimedia},
  year = {2021},
  doi = {10.1145/3474085.3475548},
}
Figure