Introduction
Multi-view clustering (MVC) has emerged as a critical technique in machine learning, designed to integrate information from diverse data sources—such as different modalities of an image or various textual descriptions of a single entity—to discover underlying group structures. Despite its potential, traditional deep MVC methods face two significant bottlenecks: the quadratic computational complexity ($O(n^2)$) associated with full-sample graph construction and the reliance on static, often suboptimal, anchors for data approximation.
This paper introduces Deep Multi-view Anchor Clustering (DMAC), a novel framework presented at AAAI 2025. DMAC addresses these challenges by transitioning from static to learnable anchors, achieving state-of-the-art performance with linear computational complexity.
Methodology: The DMAC Framework
The core innovation of DMAC lies in its ability to optimize representative "anchors" dynamically during the training process, rather than relying on fixed points.
Perturbation-Driven Anchor Learning
In traditional anchor-based methods, anchors are typically initialized via k-means and remain fixed, which limits the model's ability to adapt to the latent feature space as it evolves. DMAC introduces a Perturbation-Driven Anchor Learning mechanism. By injecting "positive-incentive noise" (sampled from a Gaussian distribution) into the initial anchors, the model makes the anchor selection process differentiable. This allows the anchors to be updated via backpropagation using a specialized Anchor Learning Loss, ensuring a clear and unambiguous mapping between data samples and their corresponding anchors.
Anchor Graph Convolution (AGC)
To capture the high-level structural relationships within the data, DMAC employs Anchor Graph Convolution. Instead of building a graph between all $n$ samples, the model constructs a graph based on the $m$ anchors. This approach allows the network to learn the topological "shape" of the data distribution efficiently, facilitating better cluster separation.
Consistency via Mutual Information Maximization
A fundamental goal of MVC is to find a consensus across multiple views. DMAC achieves this through Mutual Information (MI) maximization. Crucially, rather than aligning every individual sample across views—which is computationally expensive—DMAC aligns the probability distributions of the anchors. This anchor-level consensus mechanism is significantly faster and more robust to view-specific noise, focusing the model on shared semantic information.
Theoretical and Computational Advantages
The authors provide a rigorous theoretical foundation for DMAC. Theorem 1 in the paper demonstrates that the anchor learning loss acts as a regularization term, effectively preventing "representation collapse"—a common issue in deep clustering where the model maps all inputs to a single point in the latent space.
From a computational standpoint, DMAC reduces the complexity from $O(n^2)$ to $O(nm)$, where $m$ is the number of anchors. Since $m \ll n$, the model scales linearly with the number of samples, making it suitable for large-scale applications where traditional graph-based deep models fail.
Experimental Evaluation
The DMAC framework was evaluated against nine state-of-the-art competitors across six diverse datasets, including Yale, PIE, ALOI (images), BBC, NUS (text/tags), and CCV (video).
Performance Highlights
- Accuracy: DMAC consistently outperformed both shallow and deep MVC methods. On the Yale face dataset, it achieved an accuracy of 78.18%, surpassing the next best deep model, DealMVC (75.18%).
- Efficiency: Runtime analysis confirmed that DMAC is the fastest among deep models, maintaining a low, linear time profile even as dataset sizes increased.
- Ablation Studies: Experiments showed that removing the learnable aspect of the anchors (the perturbation mechanism) led to a sharp drop in performance, validating the importance of adaptive anchor optimization.
Conclusion and Future Directions
DMAC represents a significant step forward in making deep multi-view clustering both scalable and effective. By transforming anchors into learnable components and leveraging mutual information at the anchor level, the framework overcomes the limitations of static approximation and quadratic complexity.
While the model shows impressive results, future research may focus on reducing the sensitivity to the structure-preservation hyperparameter ($\beta$) and exploring end-to-end anchor initialization strategies that move beyond k-means. As datasets continue to grow in size and complexity, the linear-time efficiency of DMAC positions it as a robust solution for real-world multi-view data analysis.
Summary updated