Contrastive and View-Interaction Structure Learning for Multi-view Clustering

Jing Wang, Songhe Feng, Sheng-Jun Huang

Abstract

Multi-view clustering (MVC) aims to group data points by integrating information from various sources. Despite the success of Deep Multi-view Clustering (DMVC), existing methods often neglect structural information and suffer from the "false negative trap" in contrastive learning, where samples belonging to the same cluster are incorrectly penalized as negative pairs. We propose SERIES (Contrastive and viEw-interaction stRucture learning for multI-viEw cluStering), a framework that integrates sample-to-sample structural relationships and a dynamic "soft" contrastive loss. By employing Graph Convolutional Networks (GCNs) and a dynamic weight modulating function, SERIES captures local neighborhood structures and prevents the model from pushing apart similar samples. Extensive evaluations on six benchmarks show that SERIES consistently outperforms state-of-the-art methods, achieving significant gains in accuracy and robustness.

Introduction

Multi-view clustering is a critical task in data mining, leveraging complementary information from multiple modalities (e.g., images and text). While deep learning has advanced this field, two major challenges persist: the neglect of topological structural information between instances and the "False Negative Trap" in standard contrastive learning. In the latter, every sample in a batch other than the anchor is treated as a negative, even if it belongs to the same semantic cluster. This leads to increased intra-class distance and degraded clustering performance.

The SERIES Framework

The SERIES framework addresses these issues through three integrated modules: View-specific Deep Graph Autoencoders, Soft Negative Pair Aware Contrastive Learning, and Cross-view Dual Relation Generation.

The SERIES framework architecture, illustrating the flow from multi-view input to the consensus similarity matrix via GCN-based autoencoders and dual-relation generation.

Structural Representation Learning

Unlike simple encoders, SERIES utilizes Graph Convolutional Networks (GCNs) within its autoencoder structure. By ingesting both raw features ($X^v$) and a $k$-Nearest Neighbors ($k$NN) graph ($A^v$), the model ensures that latent representations ($Z^v$) capture both individual content and local neighborhood structures.

Soft Negative Pair Aware Contrastive Learning

To mitigate the false negative problem, SERIES introduces a dynamic weight modulating function ($\rho$). This function adjusts the penalty for negative pairs based on their similarity ($S$). If two samples are highly similar, they are likely in the same cluster, and the weight $\rho$ is reduced, allowing the model to pull them together rather than forcing them apart.

The Dynamic Weight Function ($\rho$) dynamically adjusts weights based on sample similarity ($S$), effectively handling "soft" negative pairs in contrastive learning.

Cross-view Dual Relation Generation

This module explores interactions between different views. It generates affinity matrices by considering both a view’s self-relation and its dual-relation with other views. These matrices are fused into a consensus similarity matrix ($\bar{S}$), which is used for final spectral clustering.

Experimental Results

SERIES was evaluated on six diverse datasets, including HW, Mfeat, Reuters, Noisyminist, VOC, and Hdigit.

Performance and Robustness

The framework consistently outperformed nine state-of-the-art baselines. On the VOC dataset, SERIES achieved a remarkable 36.25% improvement in Accuracy (ACC) over the SiMVC baseline. On the Hdigit dataset, it reached an ACC of 0.9676, surpassing the next best method by over 2%. Sensitivity analysis demonstrates that the model maintains stable performance across trade-off parameters, confirming its robustness.

Conclusion

SERIES represents a significant step forward in multi-view clustering by treating structural information as a first-class citizen and solving the false negative trap in contrastive learning. Its dynamic weighting mechanism and cross-view interaction module provide a more reliable and consistent clustering solution across varied data modalities. Future work will focus on scaling the framework to larger datasets and adapting it for incomplete multi-view scenarios.

Summary updated

BibTeX
@inproceedings{wang2024contrastive,
  author = {Jing Wang and Songhe Feng and Sheng-Jun Huang},
  title = {{Contrastive and View-Interaction Structure Learning for Multi-view Clustering}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
}
Figure