A Survey on Deep Clustering: From the Prior Perspective

Yiding Lu, Haobin Li, Yunfan Li, Yijie Lin, Xi Peng

Abstract

Facilitated by the powerful feature extraction ability of neural networks, deep clustering has achieved great success in analyzing high-dimensional and complex real-world data. The performance of deep clustering methods is affected by various factors such as network structures and learning objectives. However, as pointed out in this survey, the essence of deep clustering lies in the incorporation and utilization of prior knowledge, which is largely ignored by existing works. From pioneering deep clustering methods based on data structure assumptions to recent contrastive clustering methods based on data augmentation invariances, the development of deep clustering intrinsically corresponds to the evolution of prior knowledge. In this survey, we provide a comprehensive review of deep clustering methods by categorizing them into six types of prior knowledge. We find that in general the prior innovation follows two trends, namely, i) from mining to constructing, and ii) from internal to external. Besides, we provide a benchmark on five widely-used datasets and analyze the performance of methods with diverse priors. By providing a novel prior knowledge perspective, we hope this survey could provide some novel insights and inspire future research in the deep clustering community.

Introduction

Clustering remains a fundamental challenge in machine learning: how can data be grouped effectively without ground-truth labels? While deep learning has significantly improved feature extraction, the lack of supervision signals means neural networks often struggle to identify which features are relevant for defining clusters. Most research has focused on architectural tweaks or loss functions, but this survey argues that the core driver of success in deep clustering is the prior knowledge integrated into the models. By categorizing these priors, we can better understand the historical evolution and future trajectory of the field.

Taxonomy of Deep Clustering Priors

The authors propose a taxonomy based on six distinct categories of prior knowledge, which serve as the foundation for various deep clustering algorithms.

Taxonomy of six prior knowledge categories in deep clustering.
  1. Structure Prior: Assumes data lies on specific manifolds or subspaces, adapting classic spectral clustering concepts for the deep learning era.
  2. Distribution Prior: Assumes clusters follow specific statistical distributions, typically implemented using generative models like Variational Autoencoders (VAEs) or GANs.
  3. Augmentation Invariance: A pillar of modern contrastive learning, assuming that cluster identity should remain invariant under transformations like rotation, cropping, or color-jittering.
  4. Neighborhood Consistency: Assumes that points in close proximity within the latent feature space are likely to belong to the same semantic category.
  5. Pseudo Label: An iterative refinement process where the model's high-confidence predictions are used as self-generated supervision signals.
  6. External Knowledge: The current frontier, which utilizes pre-trained foundation models (e.g., CLIP) or textual descriptions to provide external semantic guidance.

Key Insights: The Evolution of Priors

The development of deep clustering reveals two significant trends:

  • From Mining to Constructing: Early methods focused on mining inherent data structures (like manifolds). Modern approaches construct priors, primarily through data augmentation, forcing the model to learn robust representations by comparing different "views" of the same data point.
  • From Internal to External: The field is moving beyond "closed-box" internal data analysis. Newer methods leverage external knowledge from large-scale vision-language models to provide semantic hints that internal data alone cannot provide.
  • Additive Priors: The most effective models often combine multiple priors. For instance, the SCAN algorithm integrates Augmentation Invariance for feature learning, Neighborhood Consistency for cluster assignment, and Pseudo-labeling for final refinement.

Benchmarking and Performance

The survey benchmarks performance across five image datasets: CIFAR-10, CIFAR-100, STL-10, ImageNet-10, and ImageNet-Dogs.

Mechanism of External Knowledge prior, leveraging semantic guidance.

The results indicate a clear hierarchy. Traditional methods like K-means struggle on complex datasets (e.g., ~23% accuracy on CIFAR-10). However, methods utilizing External Knowledge or advanced Augmentation Invariance achieve near-supervised performance levels. Notably, the TAC method, which leverages external knowledge, achieves 99.2% accuracy on ImageNet-10 and 83.0% on the fine-grained ImageNet-Dogs dataset, demonstrating the power of semantic guidance.

Future Directions and Challenges

Despite recent progress, several challenges remain:

  • Fine-grained Clustering: Distinguishing between highly similar categories (e.g., specific bird species) remains difficult for unsupervised models.
  • Automatic Cluster Discovery: Most methods require a pre-defined number of clusters ($C$). There is a need for non-parametric methods that can automatically determine the optimal number of clusters.
  • Fairness and Bias: Clustering algorithms can inherit and amplify biases present in the data. Developing fair clustering methods that maintain accuracy while ignoring sensitive attributes is a critical ethical priority.
  • Multi-view Integration: Future research should focus on integrating data from multiple sensors (e.g., RGB, Depth, Thermal) for more robust clustering in real-world applications like robotics.

Summary updated

Abstract

Facilitated by the powerful feature extraction ability of neural networks, deep clustering has achieved great success in analyzing high-dimensional and complex real-world data. The performance of deep clustering methods is affected by various factors such as network structures and learning objectives. However, as pointed out in this survey, the essence of deep clustering lies in the incorporation and utilization of prior knowledge, which is largely ignored by existing works. From pioneering deep clustering methods based on data structure assumptions to recent contrastive clustering methods based on data augmentation invariances, the development of deep clustering intrinsically corresponds to the evolution of prior knowledge. In this survey, we provide a comprehensive review of deep clustering methods by categorizing them into six types of prior knowledge. We find that in general the prior innovation follows two trends, namely, i) from mining to constructing, and ii) from internal to external. Besides, we provide a benchmark on five widely-used datasets and analyze the performance of methods with diverse priors. By providing a novel prior knowledge perspective, we hope this survey could provide some novel insights and inspire future research in the deep clustering community.
BibTeX
@inproceedings{lu2024survey,
  author = {Yiding Lu and Haobin Li and Yunfan Li and Yijie Lin and Xi Peng},
  title = {{A Survey on Deep Clustering: From the Prior Perspective}},
  booktitle = {Vicinagearth},
  year = {2024},
  doi = {10.1007/s44336-024-00001-w},
}
Figure