Introduction
Multiview clustering is a critical task in machine learning where data instances are described by multiple feature sets, such as visual, audio, and textual components of a video. The goal is to identify a consensus structure that captures the shared information across all views. Traditional graph-based methods often rely on spectral clustering, which involves computationally expensive eigen-decomposition ($O(n^3)$) and is sensitive to the quality of the initial static graph.
To address these challenges, this paper proposes Stationary Diffusion State Neural Estimation (SDSNE). This framework reformulates the search for an ideal clustering structure as a neural estimation problem, leveraging gradient descent to find a "stationary diffusion state" that reveals the underlying clusters without the need for explicit eigen-decomposition or traditional auto-encoders.
Methodology: Diffusion as Neural Estimation
The core innovation of SDSNE lies in its treatment of the clustering process as a Markov chain diffusion. The intuition is that in a stationary state, nodes within the same cluster should reach an equilibrium of information flow.
Key Components
- Diffusion Intuition: By modeling the graph as a system of connected vessels, the stationary state represents a configuration where connected nodes in the same cluster have identical representations. SDSNE aims to learn a cluster indicator matrix where intra-cluster nodes are fully connected with equal weights.
- Neural Diffusion Layer: Instead of standard Graph Convolutional Networks (GCNs), SDSNE defines a neural layer $H^{(v)} \leftarrow P^{(v)} W (P^{(v)})^\top$. Here, $P^{(v)}$ is the transition matrix for a specific view, and $W$ is a learnable parameter matrix shared across all views. This shared $W$ acts as a universal self-attentional module that extracts the consensus structure.
- Co-supervised Learning: Unlike many unsupervised models that use feature reconstruction (auto-encoders), SDSNE employs a co-supervised strategy. The structural information from each individual view supervises the learning of the global consensus. The loss function minimizes the discrepancy between the learned representation and the stationary state across all views.
- Optimization over Decomposition: By using backpropagation and optimizers like Adam, SDSNE directly learns the cluster indicator matrix. While the theoretical complexity remains $O(n^3)$, these matrix operations are highly parallelizable on modern GPUs, offering practical advantages over traditional eigen-solvers.
Experimental Results
The authors evaluated SDSNE on six benchmark datasets spanning text and image domains: BBC Sport, MSRC-v1, 100 Leaves, Three Sources, Scene-15, and Reuters.
SDSNE consistently outperformed state-of-the-art (SOTA) baselines such as GMC, CGD, and O2MAC. Notable results include:
- Accuracy Improvements: On the "Three Sources" dataset, SDSNE achieved 93.5% accuracy, representing a 15.4% improvement over the next best baseline.
- Scalability and Versatility: The model showed robust performance on both small-scale datasets (MSRC-v1) and large-scale multilingual datasets (Reuters).
- Structural Clarity: Qualitative analysis through t-SNE and graph heatmaps confirmed that the learned consensus graph exhibits a clear block-diagonal structure, indicating successful cluster separation.
Conclusion
SDSNE presents a novel perspective on multiview clustering by framing the stationary diffusion state as an optimization target for neural networks. By bypassing expensive eigen-decomposition and utilizing a shared self-attentional mechanism, it effectively captures consensus information across diverse data views. While $O(n^3)$ complexity remains a consideration for massive graphs, the model's parallelizability and superior accuracy mark a significant step forward in unsupervised multiview learning. Future work may explore more efficient sampling techniques or extending this framework to semi-supervised settings.
Summary updated