Continual Multi-View Clustering with Consistent Anchor Guidance

Chao Zhang, Xu Deng, Xiuyi Jia, Chunlin Chen, Huaxiong Li

Abstract

Multi-view clustering (MVC) is essential for analyzing data from diverse sources, yet traditional static methods struggle with the computational demands and privacy constraints of streaming data. We introduce ACMVC (Consistent Anchor-guided Continual Multi-View Clustering), a framework designed for efficient clustering of multi-view data streams. ACMVC utilizes a two-stage process—Initial Learning (I-Stage) and Continual Learning (C-Stage)—to transfer knowledge via multi-level anchors and consistency regularization. This approach achieves linear time and space complexity while preventing catastrophic forgetting. Experimental results across seven benchmarks, including large-scale datasets like VGGFace, demonstrate that ACMVC provides state-of-the-art accuracy and superior efficiency compared to existing online and static MVC methods.

Introduction

In real-world scenarios, data often arrives as a continuous stream from multiple sources, such as social media feeds or sensor networks. Traditional Multi-view clustering (MVC) methods typically assume the entire dataset is available upfront, making them ill-suited for dynamic environments due to high re-training costs, the risk of catastrophic forgetting, and data privacy regulations.

To address these challenges, we propose ACMVC. Our framework avoids expensive re-training by leveraging "anchors"—representative points that summarize data distributions—as knowledge carriers. By maintaining consistency between historical and current anchor embeddings, ACMVC effectively captures the evolving structure of data streams without needing to store raw historical data.

Methodology

ACMVC operates through a structured two-stage learning process designed to maintain semantic stability over time.

Initial Learning Stage (I-Stage)

Upon receiving the first data chunk, the model constructs a low-rank anchor graph. Instead of an $O(n^2)$ full-sample comparison, it identifies a set of representative anchors. The resulting shared similarity matrix $Z_1$ is factorized into sample embeddings $F_1$ and anchor embeddings $V_1^T$. This factorization captures the latent semantic relationships within the initial data.

Continual Learning Stage (C-Stage)

For subsequent data chunks, ACMVC employs Consistent Anchor Guidance. Rather than starting from scratch, the model reuses anchors and embeddings from the previous stage to guide current learning. A consistency regularization term, weighted by parameter beta, ensures that new embeddings do not drift significantly from historical ones, preserving the global cluster structure.

Figure 1: The ACMVC framework overview, illustrating the transition from the Initial Stage (I-Stage) to the Continual Stage (C-Stage) using shared weights and knowledge transfer.

The optimization process uses an alternate search strategy, often reducing sub-problems to Singular Value Decomposition (SVD) for rapid convergence.

Experiments and Results

We evaluated ACMVC on seven benchmarks, ranging from news articles (BBCSport) to large-scale face images (VGGFace). ACMVC consistently outperformed competing online methods. For instance, on the Mfeat dataset, ACMVC achieved 86.18% accuracy, a significant improvement over the 55.54% achieved by OMVC.

Efficiency and Convergence

A key advantage of ACMVC is its linear complexity $O(n)$, making it highly scalable. On the VGGFace dataset (34,000+ samples), the task was completed in under 50 seconds. The optimization algorithm demonstrates excellent stability, typically converging within 10 iterations.

Figure 3: Convergence curves of the objective function for the BBCSport and VGGFace datasets, showing rapid stabilization in both I-Stage and C-Stage.

Conclusion

ACMVC provides a robust solution for continual multi-view clustering by using anchors as efficient knowledge bridges. By combining multi-level guidance with linear complexity, it addresses the core hurdles of streaming data analysis. Future work will focus on adapting the model to handle dynamic cluster counts and automating hyperparameter selection to further enhance its real-world applicability.

Summary updated

BibTeX
@inproceedings{zhang2024continual,
  author = {Chao Zhang and Xu Deng and Xiuyi Jia and Chunlin Chen and Huaxiong Li},
  title = {{Continual Multi-View Clustering with Consistent Anchor Guidance}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
  url = {https://openreview.net/forum?id=5ZGxF6HvBj},
}
Figure